Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 929477 Details for
Bug 1132833
VDSM does not report parameters such as memUsage
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm log of host
vdsm.log (text/x-log), 10.26 MB, created by
Lukas Svaty
on 2014-08-22 08:21:17 UTC
(
hide
)
Description:
vdsm log of host
Filename:
MIME Type:
Creator:
Lukas Svaty
Created:
2014-08-22 08:21:17 UTC
Size:
10.26 MB
patch
obsolete
>Dummy-170::DEBUG::2014-08-22 09:01:03,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:03,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118185 s, 86.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:04,425::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:04,425::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:04,425::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:04,425::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:04,425::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) (190247) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) (190247) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) (190247) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:04,426::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:01:04,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:01:04,456::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000270571 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:04,456::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:04,456::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:04,457::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:04,457::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:04,457::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:04,457::__init__::225::IOProcess::(_processLogs) (190248) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:04,457::__init__::225::IOProcess::(_processLogs) (190248) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:04,457::__init__::225::IOProcess::(_processLogs) (190248) Building response >Thread-14::DEBUG::2014-08-22 09:01:04,458::__init__::225::IOProcess::(_processLogs) (190248) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:04,458::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:01:05,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:05,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119862 s, 85.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:05,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:05,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:05,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:01:06,096::task::595::Storage.TaskManager.Task::(_updateState) Task=`d75a3be0-bd23-4aab-8c4f-0b8016e1c3b3`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:06,097::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:01:06,097::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000270571', 'lastCheck': '1.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000254757', 'lastCheck': '8.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:06,097::task::1191::Storage.TaskManager.Task::(prepare) Task=`d75a3be0-bd23-4aab-8c4f-0b8016e1c3b3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000270571', 'lastCheck': '1.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000254757', 'lastCheck': '8.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:06,097::task::595::Storage.TaskManager.Task::(_updateState) Task=`d75a3be0-bd23-4aab-8c4f-0b8016e1c3b3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:06,097::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:06,097::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:06,097::task::993::Storage.TaskManager.Task::(_decref) Task=`d75a3be0-bd23-4aab-8c4f-0b8016e1c3b3`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:01:06,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:07,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:07,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118889 s, 86.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) (190249) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) (190249) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) (190249) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:07,722::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:07,723::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:01:07,723::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:01:07,752::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000222975 s, 2.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:07,755::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:07,756::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:07,757::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:07,758::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:07,759::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:07,760::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:07,760::__init__::225::IOProcess::(_processLogs) (190250) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:07,761::__init__::225::IOProcess::(_processLogs) (190250) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:07,762::__init__::225::IOProcess::(_processLogs) (190250) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:07,762::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:07,763::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:01:07,768::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:01:07,769::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:07,769::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:07,771::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:07,771::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:07,771::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:07,771::__init__::225::IOProcess::(_processLogs) (190251) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:07,771::__init__::225::IOProcess::(_processLogs) (190251) Building response >Thread-14::DEBUG::2014-08-22 09:01:07,772::__init__::225::IOProcess::(_processLogs) (190251) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:07,772::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:07,772::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:07,772::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:01:07,772::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:07,772::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) (190252) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) (190252) Building response >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) (190252) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:01:07,773::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) (190253) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) (190253) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) (190253) Building response >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) (190253) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:07,774::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:01:08,081::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:01:08,082::task::595::Storage.TaskManager.Task::(_updateState) Task=`c48c479a-61e2-42cb-8b42-eed9cb85a284`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:08,082::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:01:08,083::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:08,084::task::1191::Storage.TaskManager.Task::(prepare) Task=`c48c479a-61e2-42cb-8b42-eed9cb85a284`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:08,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`c48c479a-61e2-42cb-8b42-eed9cb85a284`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:08,085::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:08,085::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:08,086::task::993::Storage.TaskManager.Task::(_decref) Task=`c48c479a-61e2-42cb-8b42-eed9cb85a284`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:01:08,141::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:01:08,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`6358d815-4c51-4282-8ea0-657876d11d1b`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:01:08,141::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:01:08,141::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3506a072-4632-4705-83f4-c609843b4f26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:01:08,141::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:01:08,141::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:01:08,142::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3506a072-4632-4705-83f4-c609843b4f26`::Granted request >Thread-171::DEBUG::2014-08-22 09:01:08,142::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6358d815-4c51-4282-8ea0-657876d11d1b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:01:08,142::task::993::Storage.TaskManager.Task::(_decref) Task=`6358d815-4c51-4282-8ea0-657876d11d1b`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:01:08,144::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:08,144::task::1191::Storage.TaskManager.Task::(prepare) Task=`6358d815-4c51-4282-8ea0-657876d11d1b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:08,144::task::595::Storage.TaskManager.Task::(_updateState) Task=`6358d815-4c51-4282-8ea0-657876d11d1b`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:01:08,144::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:01:08,144::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:01:08,144::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:01:08,145::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:01:08,145::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:01:08,145::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:01:08,145::task::993::Storage.TaskManager.Task::(_decref) Task=`6358d815-4c51-4282-8ea0-657876d11d1b`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:09,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:09,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118134 s, 86.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:10,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:10,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:10,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:11,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:11,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0144228 s, 71.0 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:01:12,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c29471c-1b12-4f3b-bf95-908c141c1e8b`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:01:12,373::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:01:12,373::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:12,373::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:01:12,373::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:12,373::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:12,373::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:12,374::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:12,374::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:12,374::__init__::225::IOProcess::(_processLogs) (190254) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:01:12,374::__init__::225::IOProcess::(_processLogs) (190254) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:01:12,374::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|(190254) Building response >Thread-14::DEBUG::2014-08-22 09:01:12,375::__init__::225::IOProcess::(_processLogs) (190254) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:12,375::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:01:12,375::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:01:12,375::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:12,375::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190255) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190255) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190255) Building response >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) (190255) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:12,376::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:12,377::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:01:12,377::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:01:12,377::task::1191::Storage.TaskManager.Task::(prepare) Task=`3c29471c-1b12-4f3b-bf95-908c141c1e8b`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:01:12,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c29471c-1b12-4f3b-bf95-908c141c1e8b`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:01:12,377::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:01:12,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:01:12,377::task::993::Storage.TaskManager.Task::(_decref) Task=`3c29471c-1b12-4f3b-bf95-908c141c1e8b`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:13,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:13,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120221 s, 85.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) (190253) Building responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:14,459::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:14,460::__init__::225::IOProcess::(_processLogs) (190256) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:14,460::__init__::225::IOProcess::(_processLogs) (190256) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:14,460::__init__::225::IOProcess::(_processLogs) (190256) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:14,460::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:14,460::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:01:14,460::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:01:14,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000205131 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) (190257) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:14,490::__init__::225::IOProcess::(_processLogs) (190257) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:14,491::__init__::225::IOProcess::(_processLogs) (190257) Building response >Thread-14::DEBUG::2014-08-22 09:01:14,491::__init__::225::IOProcess::(_processLogs) (190257) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:14,491::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:14,491::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:15,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:15,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:15,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:15,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:15,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119309 s, 85.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:01:17,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:17,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113889 s, 89.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:17,796::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:17,797::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:17,799::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:17,800::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:17,801::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:17,803::__init__::225::IOProcess::(_processLogs) (190258) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:17,804::__init__::225::IOProcess::(_processLogs) (190258) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:17,806::__init__::225::IOProcess::(_processLogs) (190258) Building response >Thread-14::DEBUG::2014-08-22 09:01:17,807::__init__::225::IOProcess::(_processLogs) (190258) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:17,809::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:17,810::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:01:17,817::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:01:17,847::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199714 s, 2.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:17,849::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:17,850::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:17,851::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:17,851::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:17,852::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:17,853::__init__::225::IOProcess::(_processLogs) (190259) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:17,854::__init__::225::IOProcess::(_processLogs) (190259) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:17,854::__init__::225::IOProcess::(_processLogs) (190259) Building response >Thread-14::DEBUG::2014-08-22 09:01:17,855::__init__::225::IOProcess::(_processLogs) (190259) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:17,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:17,856::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:01:17,860::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:01:17,860::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:17,861::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:17,861::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:17,862::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:17,863::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:17,863::__init__::225::IOProcess::(_processLogs) (190260) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:17,864::__init__::225::IOProcess::(_processLogs) (190260) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:17,864::__init__::225::IOProcess::(_processLogs) (190260) Building response >Thread-14::DEBUG::2014-08-22 09:01:17,865::__init__::225::IOProcess::(_processLogs) (190260) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:17,865::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:17,866::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:17,868::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:01:17,869::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:17,869::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:17,870::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:17,870::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:17,871::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:17,872::__init__::225::IOProcess::(_processLogs) (190261) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:17,872::__init__::225::IOProcess::(_processLogs) (190261) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:17,873::__init__::225::IOProcess::(_processLogs) (190261) Building response >Thread-14::DEBUG::2014-08-22 09:01:17,873::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:17,874::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:17,876::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:01:17,877::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:17,878::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:17,878::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:17,879::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:17,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:17,880::__init__::225::IOProcess::(_processLogs) (190262) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:17,881::__init__::225::IOProcess::(_processLogs) (190262) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:17,881::__init__::225::IOProcess::(_processLogs) (190262) Building response >Thread-14::DEBUG::2014-08-22 09:01:17,882::__init__::225::IOProcess::(_processLogs) (190262) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:17,883::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:17,883::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:01:18,192::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [739d4759] >Thread-13::DEBUG::2014-08-22 09:01:18,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`d076fc29-34cd-411f-aa6c-e70495c38e0d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:18,194::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:01:18,195::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:18,196::task::1191::Storage.TaskManager.Task::(prepare) Task=`d076fc29-34cd-411f-aa6c-e70495c38e0d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:18,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`d076fc29-34cd-411f-aa6c-e70495c38e0d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:18,196::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:18,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:18,197::task::993::Storage.TaskManager.Task::(_decref) Task=`d076fc29-34cd-411f-aa6c-e70495c38e0d`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:01:18,226::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [739d4759] >Thread-171::DEBUG::2014-08-22 09:01:18,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`fce52122-04ba-4baf-8646-a7cd90a88fa3`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:01:18,226::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:01:18,227::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9d4e5f7e-e54d-4658-92ba-0237154d243e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:01:18,227::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:01:18,227::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:01:18,227::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9d4e5f7e-e54d-4658-92ba-0237154d243e`::Granted request >Thread-171::DEBUG::2014-08-22 09:01:18,227::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fce52122-04ba-4baf-8646-a7cd90a88fa3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:01:18,227::task::993::Storage.TaskManager.Task::(_decref) Task=`fce52122-04ba-4baf-8646-a7cd90a88fa3`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:01:18,229::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:18,230::task::1191::Storage.TaskManager.Task::(prepare) Task=`fce52122-04ba-4baf-8646-a7cd90a88fa3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:18,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`fce52122-04ba-4baf-8646-a7cd90a88fa3`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:01:18,230::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:01:18,230::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:01:18,230::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:01:18,230::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:01:18,230::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:01:18,230::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:01:18,230::task::993::Storage.TaskManager.Task::(_decref) Task=`fce52122-04ba-4baf-8646-a7cd90a88fa3`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:19,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:19,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021392 s, 47.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:20,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:20,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:20,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:01:21,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6f3f785-6591-4938-bdde-51d4861f54e1`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:21,392::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:01:21,392::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205131', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199714', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:21,392::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6f3f785-6591-4938-bdde-51d4861f54e1`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205131', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199714', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:21,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6f3f785-6591-4938-bdde-51d4861f54e1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:21,392::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:21,392::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:21,392::task::993::Storage.TaskManager.Task::(_decref) Task=`a6f3f785-6591-4938-bdde-51d4861f54e1`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:01:21,417::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:22,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:22,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130671 s, 78.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:01:24,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:24,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117063 s, 87.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:24,492::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:24,492::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:24,492::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:24,492::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:24,492::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:24,493::__init__::225::IOProcess::(_processLogs) (190263) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:24,493::__init__::225::IOProcess::(_processLogs) (190263) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:24,493::__init__::225::IOProcess::(_processLogs) (190263) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:24,493::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:24,493::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:01:24,493::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:01:24,522::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000225587 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) (190264) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) (190264) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) (190264) Building response >Thread-14::DEBUG::2014-08-22 09:01:24,523::__init__::225::IOProcess::(_processLogs) (190264) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:24,524::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:25,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:25,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:25,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:26,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:26,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124059 s, 82.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:27,885::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) (190265) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) (190265) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190265) Building response >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) (190265) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:27,886::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:27,887::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:01:27,887::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:01:27,915::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199151 s, 2.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:27,916::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:27,916::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:27,916::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:27,916::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:27,916::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) (190266) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) (190266) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190266) Building response >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) (190266) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:27,917::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:01:27,918::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:27,918::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:01:27,918::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:27,918::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:27,918::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) (190267) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) (190267) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) (190267) Building response >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) (190267) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:27,919::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) (190268) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) (190268) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) (190268) Building response >Thread-14::DEBUG::2014-08-22 09:01:27,920::__init__::225::IOProcess::(_processLogs) (190268) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:27,921::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) (190269) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) (190269) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) (190269) Building response >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) (190269) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:27,922::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:01:28,279::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:01:28,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`99eaed51-9c66-4112-921c-f58742a77e2f`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:28,280::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:01:28,280::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:28,280::task::1191::Storage.TaskManager.Task::(prepare) Task=`99eaed51-9c66-4112-921c-f58742a77e2f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:28,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`99eaed51-9c66-4112-921c-f58742a77e2f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:28,280::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:28,280::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:28,280::task::993::Storage.TaskManager.Task::(_decref) Task=`99eaed51-9c66-4112-921c-f58742a77e2f`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:01:28,305::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:01:28,305::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3147eeb-ee55-469a-a549-75d0a94de592`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:01:28,306::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:01:28,307::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2b617053-9aae-4db7-8550-61d265420b94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:01:28,307::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Dummy-170::DEBUG::2014-08-22 09:01:28,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-171::DEBUG::2014-08-22 09:01:28,309::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:01:28,338::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2b617053-9aae-4db7-8550-61d265420b94`::Granted request >Thread-171::DEBUG::2014-08-22 09:01:28,340::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a3147eeb-ee55-469a-a549-75d0a94de592`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:01:28,340::task::993::Storage.TaskManager.Task::(_decref) Task=`a3147eeb-ee55-469a-a549-75d0a94de592`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:01:28,355::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:28,356::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3147eeb-ee55-469a-a549-75d0a94de592`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:28,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3147eeb-ee55-469a-a549-75d0a94de592`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:01:28,358::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:01:28,358::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:01:28,360::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:01:28,361::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:01:28,362::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:01:28,362::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:01:28,364::task::993::Storage.TaskManager.Task::(_decref) Task=`a3147eeb-ee55-469a-a549-75d0a94de592`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:28,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0291391 s, 35.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:01:30,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:30,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115624 s, 88.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:30,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:30,627::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:30,627::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:32,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:32,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132997 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:34,525::__init__::225::IOProcess::(_processLogs) (190270) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:34,526::__init__::225::IOProcess::(_processLogs) (190270) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:34,546::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190270) Building response >Thread-14::DEBUG::2014-08-22 09:01:34,546::__init__::225::IOProcess::(_processLogs) (190270) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:34,546::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:01:34,547::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:01:34,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:01:34,582::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.0025497 s, 156 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) (190271) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:34,583::__init__::225::IOProcess::(_processLogs) (190271) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:34,587::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190271) Building response >Thread-14::DEBUG::2014-08-22 09:01:34,588::__init__::225::IOProcess::(_processLogs) (190271) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:34,588::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:01:34,588::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:01:34,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:34,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121265 s, 84.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:35,629::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:35,629::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:35,630::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:01:36,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a1dd30b-2c75-473a-952c-1d21724c2c95`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:36,679::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:01:36,679::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0025497', 'lastCheck': '2.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199151', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:36,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a1dd30b-2c75-473a-952c-1d21724c2c95`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0025497', 'lastCheck': '2.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199151', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:36,680::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a1dd30b-2c75-473a-952c-1d21724c2c95`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:36,680::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:36,680::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:36,680::task::993::Storage.TaskManager.Task::(_decref) Task=`8a1dd30b-2c75-473a-952c-1d21724c2c95`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:36,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-13::DEBUG::2014-08-22 09:01:36,733::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:36,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113515 s, 90.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:37,923::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:37,923::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:37,923::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:37,923::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) (190272) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) (190272) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190272) Building response >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) (190272) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:37,924::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:01:37,925::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:01:37,953::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000235702 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) (190273) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) (190273) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:37,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190273) Building response >Thread-14::DEBUG::2014-08-22 09:01:37,955::__init__::225::IOProcess::(_processLogs) (190273) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:37,955::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:37,955::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:01:37,955::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) (190274) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) (190274) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) (190274) Building response >Thread-14::DEBUG::2014-08-22 09:01:37,956::__init__::225::IOProcess::(_processLogs) (190274) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:37,957::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) (190275) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) (190275) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) (190275) Building response >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) (190275) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:37,958::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) (190276) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) (190276) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) (190276) Building response >Thread-14::DEBUG::2014-08-22 09:01:37,959::__init__::225::IOProcess::(_processLogs) (190276) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:37,960::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:37,960::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:01:38,451::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:01:38,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5c4e7d1-c271-4764-9533-4a07119bbda6`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:38,451::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:01:38,451::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:38,451::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5c4e7d1-c271-4764-9533-4a07119bbda6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:38,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5c4e7d1-c271-4764-9533-4a07119bbda6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:38,452::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:38,452::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:38,452::task::993::Storage.TaskManager.Task::(_decref) Task=`b5c4e7d1-c271-4764-9533-4a07119bbda6`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:01:38,510::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:01:38,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2aeff9b-f353-473d-ab62-c9a9b0510cc3`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:01:38,510::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:01:38,511::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2ae7198d-cad4-49b6-8480-d7a855a0d894`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:01:38,511::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:01:38,511::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:01:38,511::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2ae7198d-cad4-49b6-8480-d7a855a0d894`::Granted request >Thread-171::DEBUG::2014-08-22 09:01:38,511::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c2aeff9b-f353-473d-ab62-c9a9b0510cc3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:01:38,511::task::993::Storage.TaskManager.Task::(_decref) Task=`c2aeff9b-f353-473d-ab62-c9a9b0510cc3`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:01:38,513::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:38,513::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2aeff9b-f353-473d-ab62-c9a9b0510cc3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:38,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2aeff9b-f353-473d-ab62-c9a9b0510cc3`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:01:38,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:01:38,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:01:38,514::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:01:38,514::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:01:38,514::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:01:38,514::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:01:38,514::task::993::Storage.TaskManager.Task::(_decref) Task=`c2aeff9b-f353-473d-ab62-c9a9b0510cc3`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:38,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:38,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160762 s, 63.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:40,634::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:40,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:40,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:40,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:40,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116255 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:01:42,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:42,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116403 s, 88.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:44,589::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:01:44,589::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:44,589::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:44,589::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:01:44,589::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) (190277) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) (190277) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) (190277) Building response >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) (190277) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:44,590::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:01:44,591::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:01:44,619::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000657141 s, 604 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:44,620::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:44,620::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:44,620::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:44,621::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:44,621::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:44,621::__init__::225::IOProcess::(_processLogs) (190278) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:44,621::__init__::225::IOProcess::(_processLogs) (190278) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:44,621::__init__::225::IOProcess::(_processLogs) (190278) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:44,621::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:01:44,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:45,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119677 s, 85.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:45,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:45,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:45,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:47,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:47,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120495 s, 85.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) (190279) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) (190279) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) (190279) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:47,961::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:47,962::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:01:47,962::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:01:47,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00025618 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) (190280) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:47,991::__init__::225::IOProcess::(_processLogs) (190280) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:47,992::__init__::225::IOProcess::(_processLogs) (190280) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:47,992::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:47,992::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:01:47,992::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) (190281) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) (190281) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) (190281) Building response >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) (190281) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:47,993::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) (190282) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:47,994::__init__::225::IOProcess::(_processLogs) (190282) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:47,995::__init__::225::IOProcess::(_processLogs) (190282) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:47,995::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:47,995::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:47,995::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:01:47,995::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:47,995::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) (190283) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) (190283) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) (190283) Building response >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) (190283) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:47,996::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:01:48,570::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:01:48,570::task::595::Storage.TaskManager.Task::(_updateState) Task=`e51ba6bd-fe32-42b2-ad7a-63777b5defb7`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:48,570::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:01:48,570::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:48,570::task::1191::Storage.TaskManager.Task::(prepare) Task=`e51ba6bd-fe32-42b2-ad7a-63777b5defb7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:48,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`e51ba6bd-fe32-42b2-ad7a-63777b5defb7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:48,571::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:48,571::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:48,571::task::993::Storage.TaskManager.Task::(_decref) Task=`e51ba6bd-fe32-42b2-ad7a-63777b5defb7`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:01:48,597::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:01:48,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cb0b57b-25c0-4c46-ad2e-205ef9d0ba7f`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:01:48,597::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:01:48,598::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9e75a60a-b126-4520-bd44-0d50b72fef20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:01:48,598::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:01:48,598::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:01:48,598::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9e75a60a-b126-4520-bd44-0d50b72fef20`::Granted request >Thread-171::DEBUG::2014-08-22 09:01:48,598::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3cb0b57b-25c0-4c46-ad2e-205ef9d0ba7f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:01:48,598::task::993::Storage.TaskManager.Task::(_decref) Task=`3cb0b57b-25c0-4c46-ad2e-205ef9d0ba7f`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:01:48,600::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:48,601::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cb0b57b-25c0-4c46-ad2e-205ef9d0ba7f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:48,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cb0b57b-25c0-4c46-ad2e-205ef9d0ba7f`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:01:48,601::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:01:48,601::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:01:48,601::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:01:48,601::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:01:48,601::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:01:48,601::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:01:48,601::task::993::Storage.TaskManager.Task::(_decref) Task=`3cb0b57b-25c0-4c46-ad2e-205ef9d0ba7f`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:49,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:49,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117681 s, 87.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:50,648::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:50,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:50,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:51,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:51,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119107 s, 86.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:01:51,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b4bac37-85f7-4cfc-9c39-a051adfd0eb9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:51,912::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:01:51,912::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000657141', 'lastCheck': '7.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00025618', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:51,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b4bac37-85f7-4cfc-9c39-a051adfd0eb9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000657141', 'lastCheck': '7.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00025618', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:01:51,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b4bac37-85f7-4cfc-9c39-a051adfd0eb9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:51,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:51,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:51,912::task::993::Storage.TaskManager.Task::(_decref) Task=`9b4bac37-85f7-4cfc-9c39-a051adfd0eb9`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:01:51,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:53,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:53,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116252 s, 88.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:54,622::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:54,622::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) (190284) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) (190284) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) (190284) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:54,623::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:01:54,624::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:01:54,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000222061 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:54,652::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:01:54,652::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:54,652::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) (190285) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) (190285) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) (190285) Building response >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) (190285) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:54,653::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-170::DEBUG::2014-08-22 09:01:55,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:55,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118215 s, 86.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:55,655::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:55,655::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:01:55,656::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:01:57,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:57,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011515 s, 88.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:57,997::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:57,997::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:57,997::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) (190286) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) (190286) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) (190286) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:57,998::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:01:57,999::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:01:58,027::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000230355 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:01:58,027::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) (190287) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) (190287) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) (190287) Building response >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) (190287) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:58,028::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:58,029::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:01:58,029::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:58,029::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) (190288) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) (190288) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) (190288) Building response >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) (190288) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:58,030::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) (190289) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) (190289) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) (190289) Building response >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) (190289) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:58,031::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:01:58,032::__init__::225::IOProcess::(_processLogs) (190290) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:01:58,033::__init__::225::IOProcess::(_processLogs) (190290) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:01:58,033::__init__::225::IOProcess::(_processLogs) (190290) Building response >Thread-14::DEBUG::2014-08-22 09:01:58,033::__init__::225::IOProcess::(_processLogs) (190290) Queuing response >Thread-14::DEBUG::2014-08-22 09:01:58,033::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:01:58,033::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:01:58,648::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:01:58,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7f3c8b5-3ebc-43a9-af9d-fe9e43911414`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:01:58,649::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:01:58,649::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:58,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7f3c8b5-3ebc-43a9-af9d-fe9e43911414`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:01:58,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7f3c8b5-3ebc-43a9-af9d-fe9e43911414`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:01:58,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:01:58,649::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:01:58,649::task::993::Storage.TaskManager.Task::(_decref) Task=`c7f3c8b5-3ebc-43a9-af9d-fe9e43911414`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:01:58,675::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:01:58,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5287121-a504-4b98-aec6-124047325df5`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:01:58,675::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:01:58,675::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`90707ddf-c258-4cfb-ab55-dd7ac56bca28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:01:58,676::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:01:58,676::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:01:58,676::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`90707ddf-c258-4cfb-ab55-dd7ac56bca28`::Granted request >Thread-171::DEBUG::2014-08-22 09:01:58,676::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5287121-a504-4b98-aec6-124047325df5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:01:58,676::task::993::Storage.TaskManager.Task::(_decref) Task=`d5287121-a504-4b98-aec6-124047325df5`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:01:58,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:58,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5287121-a504-4b98-aec6-124047325df5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:01:58,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5287121-a504-4b98-aec6-124047325df5`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:01:58,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:01:58,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:01:58,679::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:01:58,679::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:01:58,679::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:01:58,679::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:01:58,679::task::993::Storage.TaskManager.Task::(_decref) Task=`d5287121-a504-4b98-aec6-124047325df5`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:01:59,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:01:59,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117694 s, 87.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:00,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:00,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:00,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:01,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:01,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117123 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:02:03,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:03,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115674 s, 88.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:04,654::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:04,654::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) (190291) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) (190291) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) (190291) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:04,655::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:02:04,656::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:02:04,684::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000238329 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) (190292) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) (190292) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) (190292) Building response >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) (190292) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:04,685::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:05,669::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:05,669::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:05,669::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:05,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:05,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011828 s, 86.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:07,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd9510a2-2fe7-4136-8469-18a96f896659`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:07,163::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:02:07,164::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000238329', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000230355', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:07,164::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd9510a2-2fe7-4136-8469-18a96f896659`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000238329', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000230355', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:07,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd9510a2-2fe7-4136-8469-18a96f896659`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:07,164::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:07,164::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:07,164::task::993::Storage.TaskManager.Task::(_decref) Task=`cd9510a2-2fe7-4136-8469-18a96f896659`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:02:07,189::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:07,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:07,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118093 s, 86.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:08,034::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:08,034::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:08,034::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:08,034::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:08,034::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:08,034::__init__::225::IOProcess::(_processLogs) (190293) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:08,035::__init__::225::IOProcess::(_processLogs) (190293) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:08,035::__init__::225::IOProcess::(_processLogs) (190293) Building response >Thread-14::DEBUG::2014-08-22 09:02:08,035::__init__::225::IOProcess::(_processLogs) (190293) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:08,035::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:08,035::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:02:08,035::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:02:08,064::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000237571 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:08,064::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:08,064::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) (190294) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) (190294) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) (190294) Building response >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) (190294) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:08,065::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:08,067::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:02:08,067::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) (190295) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) (190295) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) (190295) Building response >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) (190295) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:08,068::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) (190296) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) (190296) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) (190296) Building response >Thread-14::DEBUG::2014-08-22 09:02:08,069::__init__::225::IOProcess::(_processLogs) (190296) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:08,070::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:08,070::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:08,070::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:02:08,070::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:08,070::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:08,070::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) (190297) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) (190297) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) (190297) Building response >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) (190297) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:08,071::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:02:08,753::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:02:08,753::task::595::Storage.TaskManager.Task::(_updateState) Task=`329da095-ed5c-430a-9f6b-8e1963749f8e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:08,753::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:02:08,754::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:08,754::task::1191::Storage.TaskManager.Task::(prepare) Task=`329da095-ed5c-430a-9f6b-8e1963749f8e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:08,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`329da095-ed5c-430a-9f6b-8e1963749f8e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:08,754::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:08,754::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:08,754::task::993::Storage.TaskManager.Task::(_decref) Task=`329da095-ed5c-430a-9f6b-8e1963749f8e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:02:08,811::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:02:08,811::task::595::Storage.TaskManager.Task::(_updateState) Task=`61a8ed48-c8cd-46a5-bf46-b9b2419dc7cd`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:02:08,811::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:02:08,812::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`52052405-9392-4d0b-943b-d0c3a3ccf26a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:02:08,812::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:02:08,812::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:02:08,812::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`52052405-9392-4d0b-943b-d0c3a3ccf26a`::Granted request >Thread-171::DEBUG::2014-08-22 09:02:08,812::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`61a8ed48-c8cd-46a5-bf46-b9b2419dc7cd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:02:08,812::task::993::Storage.TaskManager.Task::(_decref) Task=`61a8ed48-c8cd-46a5-bf46-b9b2419dc7cd`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:02:08,814::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:08,815::task::1191::Storage.TaskManager.Task::(prepare) Task=`61a8ed48-c8cd-46a5-bf46-b9b2419dc7cd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:08,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`61a8ed48-c8cd-46a5-bf46-b9b2419dc7cd`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:02:08,815::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:02:08,815::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:02:08,815::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:02:08,815::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:02:08,815::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:02:08,815::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:02:08,815::task::993::Storage.TaskManager.Task::(_decref) Task=`61a8ed48-c8cd-46a5-bf46-b9b2419dc7cd`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:02:09,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:09,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119066 s, 86.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:10,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:10,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:10,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:11,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:12,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147455 s, 69.4 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:02:12,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`d32a33e2-17a8-441c-9f7f-66dc82112965`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:02:12,428::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:02:12,429::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:02:12,429::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:12,429::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:12,429::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:12,429::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:12,429::__init__::225::IOProcess::(_processLogs) (190298) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:02:12,430::__init__::225::IOProcess::(_processLogs) (190298) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:02:12,430::__init__::225::IOProcess::(_processLogs) (190298) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:12,430::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:02:12,431::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) (190299) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) (190299) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) (190299) Building response >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) (190299) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:12,432::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:02:12,433::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:02:12,433::task::1191::Storage.TaskManager.Task::(prepare) Task=`d32a33e2-17a8-441c-9f7f-66dc82112965`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:02:12,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`d32a33e2-17a8-441c-9f7f-66dc82112965`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:02:12,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:02:12,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:02:12,433::task::993::Storage.TaskManager.Task::(_decref) Task=`d32a33e2-17a8-441c-9f7f-66dc82112965`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:02:14,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:14,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120099 s, 85.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) (190300) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) (190300) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:14,687::__init__::225::IOProcess::(_processLogs) (190300) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:14,688::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:14,688::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:02:14,688::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:02:14,716::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000246963 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:14,717::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:14,717::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:14,717::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:14,717::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:14,717::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:14,718::__init__::225::IOProcess::(_processLogs) (190301) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:14,718::__init__::225::IOProcess::(_processLogs) (190301) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:14,718::__init__::225::IOProcess::(_processLogs) (190301) Building response >Thread-14::DEBUG::2014-08-22 09:02:14,718::__init__::225::IOProcess::(_processLogs) (190301) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:14,718::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:14,718::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:15,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:15,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:15,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:16,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:16,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120349 s, 85.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:18,072::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:18,072::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:18,072::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:18,072::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:18,072::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:18,073::__init__::225::IOProcess::(_processLogs) (190302) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:18,073::__init__::225::IOProcess::(_processLogs) (190302) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:18,073::__init__::225::IOProcess::(_processLogs) (190302) Building response >Thread-14::DEBUG::2014-08-22 09:02:18,073::__init__::225::IOProcess::(_processLogs) (190302) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:18,073::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-16::DEBUG::2014-08-22 09:02:18,073::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:02:18,101::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000237163 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:18,102::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:18,102::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:18,102::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) (190303) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) (190303) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) (190303) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:18,103::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) (190304) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:18,104::__init__::225::IOProcess::(_processLogs) (190304) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:18,105::__init__::225::IOProcess::(_processLogs) (190304) Building response >Thread-14::DEBUG::2014-08-22 09:02:18,105::__init__::225::IOProcess::(_processLogs) (190304) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:18,105::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:18,105::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:18,105::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:02:18,105::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) (190305) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) (190305) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) (190305) Building response >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) (190305) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:18,106::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) (190306) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) (190306) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) (190306) Building response >Thread-14::DEBUG::2014-08-22 09:02:18,107::__init__::225::IOProcess::(_processLogs) (190306) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:18,108::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:18,108::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:02:18,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:18,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119047 s, 86.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:18,891::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [2db01973] >Thread-13::DEBUG::2014-08-22 09:02:18,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`02a1ea42-6f74-4fe7-a8b8-ea35b2e28bd5`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:18,892::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:02:18,892::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:18,892::task::1191::Storage.TaskManager.Task::(prepare) Task=`02a1ea42-6f74-4fe7-a8b8-ea35b2e28bd5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:18,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`02a1ea42-6f74-4fe7-a8b8-ea35b2e28bd5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:18,892::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:18,892::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:18,892::task::993::Storage.TaskManager.Task::(_decref) Task=`02a1ea42-6f74-4fe7-a8b8-ea35b2e28bd5`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:02:18,934::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [2db01973] >Thread-171::DEBUG::2014-08-22 09:02:18,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`19a26970-dda9-4fc5-aa69-1b6310ade35a`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:02:18,935::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:02:18,935::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2a1ebd13-20c5-45af-9594-f0e6e75587e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:02:18,935::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:02:18,935::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:02:18,936::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2a1ebd13-20c5-45af-9594-f0e6e75587e6`::Granted request >Thread-171::DEBUG::2014-08-22 09:02:18,936::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`19a26970-dda9-4fc5-aa69-1b6310ade35a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:02:18,936::task::993::Storage.TaskManager.Task::(_decref) Task=`19a26970-dda9-4fc5-aa69-1b6310ade35a`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:02:18,938::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:18,938::task::1191::Storage.TaskManager.Task::(prepare) Task=`19a26970-dda9-4fc5-aa69-1b6310ade35a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:18,938::task::595::Storage.TaskManager.Task::(_updateState) Task=`19a26970-dda9-4fc5-aa69-1b6310ade35a`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:02:18,938::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:02:18,938::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:02:18,938::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:02:18,939::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:02:18,939::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:02:18,939::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:02:18,939::task::993::Storage.TaskManager.Task::(_decref) Task=`19a26970-dda9-4fc5-aa69-1b6310ade35a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:02:20,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:20,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172134 s, 59.5 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:20,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:20,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:20,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:22,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:22,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117047 s, 87.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:22,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`db9f2f67-9b3c-4e31-b2cd-14e21df1e63d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:22,461::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:02:22,461::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000246963', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237163', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:22,461::task::1191::Storage.TaskManager.Task::(prepare) Task=`db9f2f67-9b3c-4e31-b2cd-14e21df1e63d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000246963', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237163', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:22,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`db9f2f67-9b3c-4e31-b2cd-14e21df1e63d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:22,461::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:22,461::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:22,461::task::993::Storage.TaskManager.Task::(_decref) Task=`db9f2f67-9b3c-4e31-b2cd-14e21df1e63d`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:02:22,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:24,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:24,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120974 s, 84.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:24,719::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:24,719::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:24,719::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) (190307) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) (190307) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) (190307) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:24,720::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:02:24,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:02:24,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000252447 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) (190308) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) (190308) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) (190308) Building response >Thread-14::DEBUG::2014-08-22 09:02:24,750::__init__::225::IOProcess::(_processLogs) (190308) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:24,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:24,751::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:25,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:25,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:25,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:26,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:26,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194216 s, 52.7 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-22 09:02:28,108::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-14::DEBUG::2014-08-22 09:02:28,110::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-14::DEBUG::2014-08-22 09:02:28,110::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,110::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) (190309) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) (190309) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) (190309) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) (190309) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) Message size is 145 >Thread-14::DEBUG::2014-08-22 09:02:28,111::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) (190310) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) (190310) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) (190310) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,112::__init__::225::IOProcess::(_processLogs) (190310) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,113::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,113::__init__::225::IOProcess::(_processLogs) Sending response sized 195 >Thread-16::DEBUG::2014-08-22 09:02:28,115::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-14::DEBUG::2014-08-22 09:02:28,115::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,115::__init__::225::IOProcess::(_processLogs) Message size is 174 >Thread-14::DEBUG::2014-08-22 09:02:28,115::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,115::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,115::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,115::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,116::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,116::__init__::225::IOProcess::(_processLogs) (190311) Finding callback 'touch'... >Thread-14::DEBUG::2014-08-22 09:02:28,116::__init__::225::IOProcess::(_processLogs) (190311) Got request for method 'touch' >Thread-14::DEBUG::2014-08-22 09:02:28,141::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190311) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,142::__init__::225::IOProcess::(_processLogs) (190311) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,142::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,142::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-16::DEBUG::2014-08-22 09:02:28,142::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-22 09:02:28,142::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,142::__init__::225::IOProcess::(_processLogs) Message size is 198 >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) (190312) Finding callback 'readfile'... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) (190312) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190312) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) (190312) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,143::__init__::225::IOProcess::(_processLogs) Sending response sized 834 >Thread-16::DEBUG::2014-08-22 09:02:28,144::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-14::DEBUG::2014-08-22 09:02:28,144::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,144::__init__::225::IOProcess::(_processLogs) Message size is 191 >Thread-14::DEBUG::2014-08-22 09:02:28,144::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) (190313) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) (190313) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) (190313) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) (190313) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,145::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) (190314) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:02:28,146::__init__::225::IOProcess::(_processLogs) (190314) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:02:28,147::__init__::225::IOProcess::(_processLogs) (190314) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,147::__init__::225::IOProcess::(_processLogs) (190314) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,147::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,147::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-16::DEBUG::2014-08-22 09:02:28,147::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-22 09:02:28,147::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-16::INFO::2014-08-22 09:02:28,147::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) (190315) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) (190315) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:28,148::__init__::225::IOProcess::(_processLogs) (190315) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,149::__init__::225::IOProcess::(_processLogs) (190315) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,149::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,149::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:02:28,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:02:28,177::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000325234 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:28,178::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,178::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:28,178::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) (190316) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) (190316) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190316) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,179::__init__::225::IOProcess::(_processLogs) (190316) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,180::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,180::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) (190317) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) (190317) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) (190317) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,181::__init__::225::IOProcess::(_processLogs) (190317) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,182::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) (190318) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) (190318) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) (190318) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) (190318) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,183::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) (190319) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) (190319) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:28,184::__init__::225::IOProcess::(_processLogs) (190319) Building response >Thread-14::DEBUG::2014-08-22 09:02:28,185::__init__::225::IOProcess::(_processLogs) (190319) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:28,185::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:28,185::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:02:28,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:28,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122655 s, 83.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:28,999::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:02:28,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`aac9143c-7fc7-4b1c-9c9d-c8c4602c43db`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:28,999::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:02:28,999::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:29,000::task::1191::Storage.TaskManager.Task::(prepare) Task=`aac9143c-7fc7-4b1c-9c9d-c8c4602c43db`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:29,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`aac9143c-7fc7-4b1c-9c9d-c8c4602c43db`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:29,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:29,000::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:29,000::task::993::Storage.TaskManager.Task::(_decref) Task=`aac9143c-7fc7-4b1c-9c9d-c8c4602c43db`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:02:29,038::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:02:29,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`326edfdd-9cc2-4302-b20f-7762047ba2a4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:02:29,039::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:02:29,039::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`87731f61-f565-4084-9777-3df047f677eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:02:29,039::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:02:29,039::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:02:29,039::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`87731f61-f565-4084-9777-3df047f677eb`::Granted request >Thread-171::DEBUG::2014-08-22 09:02:29,040::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`326edfdd-9cc2-4302-b20f-7762047ba2a4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:02:29,040::task::993::Storage.TaskManager.Task::(_decref) Task=`326edfdd-9cc2-4302-b20f-7762047ba2a4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:02:29,042::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:29,042::task::1191::Storage.TaskManager.Task::(prepare) Task=`326edfdd-9cc2-4302-b20f-7762047ba2a4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:29,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`326edfdd-9cc2-4302-b20f-7762047ba2a4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:02:29,042::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:02:29,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:02:29,042::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:02:29,042::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:02:29,043::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:02:29,043::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:02:29,043::task::993::Storage.TaskManager.Task::(_decref) Task=`326edfdd-9cc2-4302-b20f-7762047ba2a4`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:02:30,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:30,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:30,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:30,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:30,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114058 s, 89.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:02:32,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:32,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117228 s, 87.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:34,752::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:34,752::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:34,752::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:34,752::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:34,752::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:34,752::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) (190320) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) (190320) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190320) Building response >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) (190320) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:34,753::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:02:34,754::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:02:34,782::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000296042 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:34,782::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:34,782::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) (190321) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) (190321) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) (190321) Building response >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) (190321) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:34,783::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-170::DEBUG::2014-08-22 09:02:34,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:34,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012432 s, 82.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:35,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:35,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:35,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:36,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:36,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117575 s, 87.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:37,718::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bb564a4-f7c9-4380-a40e-62db77b61d9a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:37,718::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:02:37,718::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000296042', 'lastCheck': '2.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000325234', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:37,718::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bb564a4-f7c9-4380-a40e-62db77b61d9a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000296042', 'lastCheck': '2.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000325234', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:37,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bb564a4-f7c9-4380-a40e-62db77b61d9a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:37,719::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:37,719::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:37,719::task::993::Storage.TaskManager.Task::(_decref) Task=`9bb564a4-f7c9-4380-a40e-62db77b61d9a`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:02:37,745::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) (190322) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) (190322) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:38,186::__init__::225::IOProcess::(_processLogs) (190322) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:38,187::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:38,187::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:02:38,187::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:02:38,215::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000207204 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:38,216::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:38,216::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:38,216::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:38,216::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:38,216::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:38,216::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:38,217::__init__::225::IOProcess::(_processLogs) (190323) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:38,217::__init__::225::IOProcess::(_processLogs) (190323) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:38,217::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190323) Building response >Thread-14::DEBUG::2014-08-22 09:02:38,217::__init__::225::IOProcess::(_processLogs) (190323) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:38,217::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:38,217::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:38,218::__init__::225::IOProcess::(_processLogs) (190324) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) (190324) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) (190324) Building response >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) (190324) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:38,219::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) (190325) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) (190325) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) (190325) Building response >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) (190325) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:38,220::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:38,221::__init__::225::IOProcess::(_processLogs) (190326) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:38,222::__init__::225::IOProcess::(_processLogs) (190326) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:38,222::__init__::225::IOProcess::(_processLogs) (190326) Building response >Thread-14::DEBUG::2014-08-22 09:02:38,222::__init__::225::IOProcess::(_processLogs) (190326) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:38,222::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:38,222::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:02:38,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:39,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129626 s, 79.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:39,091::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [5a785a8f] >Thread-13::DEBUG::2014-08-22 09:02:39,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f9a3684-ee4b-4a61-87b1-bf2c3bfad79f`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:39,091::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:02:39,091::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:39,092::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f9a3684-ee4b-4a61-87b1-bf2c3bfad79f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:39,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f9a3684-ee4b-4a61-87b1-bf2c3bfad79f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:39,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:39,092::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:39,092::task::993::Storage.TaskManager.Task::(_decref) Task=`4f9a3684-ee4b-4a61-87b1-bf2c3bfad79f`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:02:39,120::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [5a785a8f] >Thread-171::DEBUG::2014-08-22 09:02:39,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`95cc4489-2bcf-4f7b-a1d3-d68a95882730`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:02:39,121::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:02:39,121::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d4154e8b-658c-49bb-a0fe-4e0ffffbc5fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:02:39,121::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:02:39,121::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:02:39,121::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d4154e8b-658c-49bb-a0fe-4e0ffffbc5fd`::Granted request >Thread-171::DEBUG::2014-08-22 09:02:39,121::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`95cc4489-2bcf-4f7b-a1d3-d68a95882730`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:02:39,122::task::993::Storage.TaskManager.Task::(_decref) Task=`95cc4489-2bcf-4f7b-a1d3-d68a95882730`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:02:39,124::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:39,124::task::1191::Storage.TaskManager.Task::(prepare) Task=`95cc4489-2bcf-4f7b-a1d3-d68a95882730`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:39,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`95cc4489-2bcf-4f7b-a1d3-d68a95882730`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:02:39,124::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:02:39,124::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:02:39,124::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:02:39,124::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:02:39,124::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:02:39,124::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:02:39,125::task::993::Storage.TaskManager.Task::(_decref) Task=`95cc4489-2bcf-4f7b-a1d3-d68a95882730`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:40,731::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:40,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:40,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:41,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:41,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011501 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:02:43,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:43,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115711 s, 88.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:44,785::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:44,785::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:44,786::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:44,786::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:44,786::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:44,786::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:44,786::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:44,787::__init__::225::IOProcess::(_processLogs) (190327) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:44,787::__init__::225::IOProcess::(_processLogs) (190327) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:44,787::__init__::225::IOProcess::(_processLogs) (190327) Building response >Thread-14::DEBUG::2014-08-22 09:02:44,787::__init__::225::IOProcess::(_processLogs) (190327) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:44,787::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:44,787::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:02:44,788::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:02:44,815::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000223647 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:44,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:44,816::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:44,816::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:44,816::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:44,816::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) (190328) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) (190328) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) (190328) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:44,817::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:02:45,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:45,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119885 s, 85.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:45,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:45,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:45,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:47,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:47,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118789 s, 86.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:48,223::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:48,223::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:48,223::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:48,223::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:48,223::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:48,223::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:48,224::__init__::225::IOProcess::(_processLogs) (190329) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:48,224::__init__::225::IOProcess::(_processLogs) (190329) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:48,224::__init__::225::IOProcess::(_processLogs) (190329) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:48,224::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:48,224::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:02:48,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:02:48,252::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000245379 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:48,253::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:48,253::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:48,253::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) (190330) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) (190330) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) DEBUG|(190330) Building response >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) (190330) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:48,254::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:02:48,255::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:48,255::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:02:48,255::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:48,255::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) (190331) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) (190331) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) (190331) Building response >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) (190331) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:48,256::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) (190332) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:48,257::__init__::225::IOProcess::(_processLogs) (190332) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) (190332) Building response >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) (190332) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:02:48,258::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) (190333) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) (190333) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) (190333) Building response >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) (190333) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:48,259::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:02:49,170::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:02:49,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`460d5cfd-3582-458b-a90c-8ec6c74a9f53`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:49,171::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:02:49,171::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:49,171::task::1191::Storage.TaskManager.Task::(prepare) Task=`460d5cfd-3582-458b-a90c-8ec6c74a9f53`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:49,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`460d5cfd-3582-458b-a90c-8ec6c74a9f53`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:49,171::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:49,171::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:49,171::task::993::Storage.TaskManager.Task::(_decref) Task=`460d5cfd-3582-458b-a90c-8ec6c74a9f53`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:02:49,242::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:02:49,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4f4cb55-03c1-48fc-90a2-ea6bc01385e0`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:02:49,242::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:02:49,242::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`538a54a5-174e-4839-b804-df349f20ed74`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:02:49,243::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:02:49,243::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:02:49,243::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`538a54a5-174e-4839-b804-df349f20ed74`::Granted request >Thread-171::DEBUG::2014-08-22 09:02:49,243::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e4f4cb55-03c1-48fc-90a2-ea6bc01385e0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:02:49,243::task::993::Storage.TaskManager.Task::(_decref) Task=`e4f4cb55-03c1-48fc-90a2-ea6bc01385e0`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:02:49,245::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:49,245::task::1191::Storage.TaskManager.Task::(prepare) Task=`e4f4cb55-03c1-48fc-90a2-ea6bc01385e0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:49,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4f4cb55-03c1-48fc-90a2-ea6bc01385e0`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:02:49,245::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:02:49,245::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:02:49,246::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:02:49,246::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:02:49,246::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:02:49,246::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:02:49,246::task::993::Storage.TaskManager.Task::(_decref) Task=`e4f4cb55-03c1-48fc-90a2-ea6bc01385e0`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:02:49,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:49,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212276 s, 48.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:50,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:50,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:50,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:51,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:51,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117205 s, 87.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:02:52,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`73e8eafe-174c-4594-9481-7b957b98debc`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:52,967::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:02:52,967::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223647', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245379', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:52,967::task::1191::Storage.TaskManager.Task::(prepare) Task=`73e8eafe-174c-4594-9481-7b957b98debc`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223647', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245379', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:02:52,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`73e8eafe-174c-4594-9481-7b957b98debc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:52,967::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:52,967::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:52,967::task::993::Storage.TaskManager.Task::(_decref) Task=`73e8eafe-174c-4594-9481-7b957b98debc`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:02:52,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:53,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:53,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116096 s, 88.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:54,818::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:54,818::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:54,819::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:54,819::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:54,819::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:54,819::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:54,819::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:54,819::__init__::225::IOProcess::(_processLogs) (190334) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:54,820::__init__::225::IOProcess::(_processLogs) (190334) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:54,820::__init__::225::IOProcess::(_processLogs) (190334) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:54,820::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:54,820::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:02:54,820::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:02:54,849::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000587598 s, 676 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) (190335) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) (190335) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:54,850::__init__::225::IOProcess::(_processLogs) (190335) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:54,851::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:54,851::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:02:55,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:55,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120201 s, 85.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:55,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:55,770::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:02:55,772::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:02:57,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:57,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115612 s, 88.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:58,260::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) (190336) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) (190336) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) (190336) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:58,261::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:02:58,262::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:02:58,290::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00023244 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) (190337) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:02:58,291::__init__::225::IOProcess::(_processLogs) (190337) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:02:58,292::__init__::225::IOProcess::(_processLogs) DEBUG|(190337) Building response >Thread-14::DEBUG::2014-08-22 09:02:58,292::__init__::225::IOProcess::(_processLogs) (190337) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:58,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:58,292::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) (190338) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) (190338) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:58,293::__init__::225::IOProcess::(_processLogs) (190338) Building response >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) (190338) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:58,294::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) (190339) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) (190339) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) (190339) Building response >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) (190339) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:58,295::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) (190340) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) (190340) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:02:58,296::__init__::225::IOProcess::(_processLogs) (190340) Building response >Thread-14::DEBUG::2014-08-22 09:02:58,297::__init__::225::IOProcess::(_processLogs) (190340) Queuing response >Thread-14::DEBUG::2014-08-22 09:02:58,297::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:02:58,297::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:02:59,299::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-13::DEBUG::2014-08-22 09:02:59,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`633620c6-aa53-4bf3-9951-95206eb678f9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:02:59,299::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:02:59,299::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:59,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`633620c6-aa53-4bf3-9951-95206eb678f9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:02:59,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`633620c6-aa53-4bf3-9951-95206eb678f9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:02:59,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:02:59,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:02:59,300::task::993::Storage.TaskManager.Task::(_decref) Task=`633620c6-aa53-4bf3-9951-95206eb678f9`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:02:59,443::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-171::DEBUG::2014-08-22 09:02:59,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c7e7be0-c721-4a73-9446-beb36d195be8`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:02:59,445::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:02:59,446::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`06907f11-1d28-4c3f-81b7-434c891e3564`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:02:59,447::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:02:59,448::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:02:59,448::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`06907f11-1d28-4c3f-81b7-434c891e3564`::Granted request >Thread-171::DEBUG::2014-08-22 09:02:59,449::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8c7e7be0-c721-4a73-9446-beb36d195be8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:02:59,450::task::993::Storage.TaskManager.Task::(_decref) Task=`8c7e7be0-c721-4a73-9446-beb36d195be8`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:02:59,460::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:59,461::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c7e7be0-c721-4a73-9446-beb36d195be8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:02:59,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c7e7be0-c721-4a73-9446-beb36d195be8`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:02:59,462::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:02:59,462::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:02:59,463::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:02:59,464::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:02:59,464::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:02:59,464::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:02:59,465::task::993::Storage.TaskManager.Task::(_decref) Task=`8c7e7be0-c721-4a73-9446-beb36d195be8`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:02:59,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:02:59,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119939 s, 85.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:00,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:00,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:00,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:01,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:01,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114341 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:03:04,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:04,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116302 s, 88.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:04,852::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:04,852::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) (190341) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) (190341) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) (190341) Building response >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) (190341) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:04,853::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:03:04,854::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:03:04,882::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000253197 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:04,883::__init__::225::IOProcess::(_processLogs) (190342) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:04,884::__init__::225::IOProcess::(_processLogs) (190342) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:04,884::__init__::225::IOProcess::(_processLogs) (190342) Building response >Thread-14::DEBUG::2014-08-22 09:03:04,884::__init__::225::IOProcess::(_processLogs) (190342) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:04,884::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:04,884::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:05,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:05,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:05,787::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:06,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:06,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119922 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:03:08,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:08,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114533 s, 89.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:08,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`efe9c6e0-f117-493f-ad5d-c696d2e43ddf`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:08,214::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:03:08,215::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000253197', 'lastCheck': '3.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00023244', 'lastCheck': '9.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:08,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`efe9c6e0-f117-493f-ad5d-c696d2e43ddf`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000253197', 'lastCheck': '3.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00023244', 'lastCheck': '9.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:08,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`efe9c6e0-f117-493f-ad5d-c696d2e43ddf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:08,216::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:08,216::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:08,217::task::993::Storage.TaskManager.Task::(_decref) Task=`efe9c6e0-f117-493f-ad5d-c696d2e43ddf`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:03:08,254::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:08,298::__init__::225::IOProcess::(_processLogs) (190343) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:08,299::__init__::225::IOProcess::(_processLogs) (190343) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:08,299::__init__::225::IOProcess::(_processLogs) DEBUG|(190343) Building response >Thread-14::DEBUG::2014-08-22 09:03:08,299::__init__::225::IOProcess::(_processLogs) (190343) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:08,299::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:08,299::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:03:08,299::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:03:08,327::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00027454 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:08,328::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:08,328::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:08,328::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:08,328::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) (190344) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) (190344) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) DEBUG|(190344) Building response >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) (190344) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:08,329::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:03:08,330::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:08,330::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:03:08,330::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:08,330::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:08,330::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) (190345) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) (190345) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) (190345) Building response >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) (190345) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:08,331::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) (190346) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) (190346) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:08,332::__init__::225::IOProcess::(_processLogs) (190346) Building response >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) (190346) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:08,333::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) (190347) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) (190347) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) (190347) Building response >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) (190347) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:08,334::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:03:09,512::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:03:09,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`1341b2a5-d8a7-4555-ae09-51cf8d4ccb5c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:09,512::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:03:09,513::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:09,513::task::1191::Storage.TaskManager.Task::(prepare) Task=`1341b2a5-d8a7-4555-ae09-51cf8d4ccb5c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:09,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`1341b2a5-d8a7-4555-ae09-51cf8d4ccb5c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:09,513::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:09,513::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:09,513::task::993::Storage.TaskManager.Task::(_decref) Task=`1341b2a5-d8a7-4555-ae09-51cf8d4ccb5c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:03:09,555::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:03:09,556::task::595::Storage.TaskManager.Task::(_updateState) Task=`aae2e037-2e1f-4f40-ad08-c29dc4e516d6`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:03:09,557::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:03:09,559::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c781e4aa-1fd5-4186-a4e3-34ac771fb0cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:03:09,559::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:03:09,561::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:03:09,561::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c781e4aa-1fd5-4186-a4e3-34ac771fb0cd`::Granted request >Thread-171::DEBUG::2014-08-22 09:03:09,563::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aae2e037-2e1f-4f40-ad08-c29dc4e516d6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:03:09,564::task::993::Storage.TaskManager.Task::(_decref) Task=`aae2e037-2e1f-4f40-ad08-c29dc4e516d6`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:03:09,579::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:09,580::task::1191::Storage.TaskManager.Task::(prepare) Task=`aae2e037-2e1f-4f40-ad08-c29dc4e516d6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:09,581::task::595::Storage.TaskManager.Task::(_updateState) Task=`aae2e037-2e1f-4f40-ad08-c29dc4e516d6`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:03:09,581::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:03:09,582::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:03:09,583::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:03:09,584::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:03:09,585::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:03:09,585::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:03:09,586::task::993::Storage.TaskManager.Task::(_decref) Task=`aae2e037-2e1f-4f40-ad08-c29dc4e516d6`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:03:10,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:10,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120766 s, 84.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:10,789::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:10,790::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:10,790::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:12,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:12,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116345 s, 88.0 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:03:12,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a41d78e-d0ac-428a-b08e-d15689e3be3d`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:03:12,482::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:03:12,482::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:12,482::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) (190348) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) (190348) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) DEBUG|(190348) Building response >Thread-14::DEBUG::2014-08-22 09:03:12,483::__init__::225::IOProcess::(_processLogs) (190348) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:12,484::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:03:12,484::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:03:12,484::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:12,484::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) (190349) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) (190349) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) (190349) Building response >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) (190349) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:12,485::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:03:12,486::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:03:12,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a41d78e-d0ac-428a-b08e-d15689e3be3d`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:03:12,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a41d78e-d0ac-428a-b08e-d15689e3be3d`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:03:12,486::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:03:12,486::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:03:12,486::task::993::Storage.TaskManager.Task::(_decref) Task=`5a41d78e-d0ac-428a-b08e-d15689e3be3d`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:03:14,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:14,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117439 s, 87.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:14,885::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:14,885::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) (190350) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:14,886::__init__::225::IOProcess::(_processLogs) (190350) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:14,887::__init__::225::IOProcess::(_processLogs) (190350) Building response >Thread-14::DEBUG::2014-08-22 09:03:14,887::__init__::225::IOProcess::(_processLogs) (190350) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:14,887::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:14,887::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:03:14,887::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:03:14,915::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000304226 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:14,916::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:14,916::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:14,916::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:14,916::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:14,916::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) (190351) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) (190351) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) (190351) Building response >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) (190351) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:14,917::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:15,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:15,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:15,796::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:16,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:16,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180886 s, 56.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:18,335::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:18,335::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) (190352) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) (190352) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) DEBUG|(190352) Building response >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) (190352) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:18,336::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:03:18,337::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:03:18,365::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000226229 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:18,366::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:18,367::__init__::225::IOProcess::(_processLogs) (190353) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:18,367::__init__::225::IOProcess::(_processLogs) (190353) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:18,367::__init__::225::IOProcess::(_processLogs) (190353) Building response >Thread-14::DEBUG::2014-08-22 09:03:18,367::__init__::225::IOProcess::(_processLogs) (190353) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:18,367::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:18,367::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) (190354) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) (190354) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:18,368::__init__::225::IOProcess::(_processLogs) (190354) Building response >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) (190354) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:18,369::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) (190355) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) (190355) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) (190355) Building response >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) (190355) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:18,370::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) (190356) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) (190356) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) (190356) Building response >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) (190356) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:18,371::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:03:18,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:18,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119291 s, 85.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:19,674::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:03:19,676::task::595::Storage.TaskManager.Task::(_updateState) Task=`22800989-79cf-4370-a001-a6bf71b65ce0`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:19,678::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:03:19,680::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:19,681::task::1191::Storage.TaskManager.Task::(prepare) Task=`22800989-79cf-4370-a001-a6bf71b65ce0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:19,682::task::595::Storage.TaskManager.Task::(_updateState) Task=`22800989-79cf-4370-a001-a6bf71b65ce0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:19,683::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:19,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:19,686::task::993::Storage.TaskManager.Task::(_decref) Task=`22800989-79cf-4370-a001-a6bf71b65ce0`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:03:19,732::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:03:19,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4df130b-c0d0-451a-a16e-70059ed5e7d8`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:03:19,734::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:03:19,737::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9b2101b7-208f-4309-80eb-4625e9b5ae0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:03:19,737::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:03:19,738::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:03:19,739::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9b2101b7-208f-4309-80eb-4625e9b5ae0f`::Granted request >Thread-171::DEBUG::2014-08-22 09:03:19,741::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e4df130b-c0d0-451a-a16e-70059ed5e7d8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:03:19,742::task::993::Storage.TaskManager.Task::(_decref) Task=`e4df130b-c0d0-451a-a16e-70059ed5e7d8`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:03:19,760::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:19,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`e4df130b-c0d0-451a-a16e-70059ed5e7d8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:19,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4df130b-c0d0-451a-a16e-70059ed5e7d8`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:03:19,762::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:03:19,763::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:03:19,764::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:03:19,765::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:03:19,766::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:03:19,767::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:03:19,768::task::993::Storage.TaskManager.Task::(_decref) Task=`e4df130b-c0d0-451a-a16e-70059ed5e7d8`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:03:20,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:20,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0151927 s, 67.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:20,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:20,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:20,800::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:22,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:22,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116983 s, 87.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:23,489::task::595::Storage.TaskManager.Task::(_updateState) Task=`020ade70-9c4f-4a95-992c-1ed1651c6d8f`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:23,489::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:03:23,490::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000304226', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000226229', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:23,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`020ade70-9c4f-4a95-992c-1ed1651c6d8f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000304226', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000226229', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:23,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`020ade70-9c4f-4a95-992c-1ed1651c6d8f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:23,490::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:23,490::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:23,490::task::993::Storage.TaskManager.Task::(_decref) Task=`020ade70-9c4f-4a95-992c-1ed1651c6d8f`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:03:23,512::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:24,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:24,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117983 s, 86.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:24,918::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) (190357) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) (190357) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190357) Building response >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) (190357) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:24,919::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:03:24,920::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:03:24,948::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00020532 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:24,949::__init__::225::IOProcess::(_processLogs) (190358) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:24,950::__init__::225::IOProcess::(_processLogs) (190358) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:24,950::__init__::225::IOProcess::(_processLogs) (190358) Building response >Thread-14::DEBUG::2014-08-22 09:03:24,950::__init__::225::IOProcess::(_processLogs) (190358) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:24,950::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:24,950::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:25,805::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:25,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:25,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:26,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:26,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124395 s, 82.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:28,372::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) (190359) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) (190359) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190359) Building response >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) (190359) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:28,373::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:28,374::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:03:28,374::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:03:28,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223757 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:28,403::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:28,403::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:28,403::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:28,403::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:28,403::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) (190360) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) (190360) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) (190360) Building response >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) (190360) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:28,404::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:03:28,405::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:28,405::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:03:28,405::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:28,405::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:28,405::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) (190361) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) (190361) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) (190361) Building response >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) (190361) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:28,406::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) (190362) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) (190362) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) (190362) Building response >Thread-14::DEBUG::2014-08-22 09:03:28,407::__init__::225::IOProcess::(_processLogs) (190362) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:28,408::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) (190363) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) (190363) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) (190363) Building response >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) (190363) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:28,409::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:03:28,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:28,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013152 s, 77.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:29,851::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:03:29,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7cbd445-2e62-486b-938f-f1f2235dff44`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:29,851::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:03:29,851::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:29,851::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7cbd445-2e62-486b-938f-f1f2235dff44`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:29,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7cbd445-2e62-486b-938f-f1f2235dff44`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:29,852::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:29,852::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:29,852::task::993::Storage.TaskManager.Task::(_decref) Task=`f7cbd445-2e62-486b-938f-f1f2235dff44`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:03:29,885::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:03:29,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`c018c163-5072-450c-9279-01c4213356a2`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:03:29,886::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:03:29,886::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`46313ff1-3fde-4eed-8097-6e645664a18d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:03:29,886::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:03:29,886::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:03:29,886::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`46313ff1-3fde-4eed-8097-6e645664a18d`::Granted request >Thread-171::DEBUG::2014-08-22 09:03:29,887::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c018c163-5072-450c-9279-01c4213356a2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:03:29,887::task::993::Storage.TaskManager.Task::(_decref) Task=`c018c163-5072-450c-9279-01c4213356a2`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:03:29,889::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:29,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`c018c163-5072-450c-9279-01c4213356a2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:29,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`c018c163-5072-450c-9279-01c4213356a2`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:03:29,889::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:03:29,889::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:03:29,889::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:03:29,889::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:03:29,889::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:03:29,889::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:03:29,890::task::993::Storage.TaskManager.Task::(_decref) Task=`c018c163-5072-450c-9279-01c4213356a2`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:30,812::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:30,812::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:30,813::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:30,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:31,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121245 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:03:33,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:33,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011571 s, 88.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:34,951::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:34,951::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:34,951::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) (190364) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) (190364) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) (190364) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:34,952::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:03:34,953::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:03:34,981::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000191569 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:34,981::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:34,981::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) (190365) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) (190365) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) (190365) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:34,982::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:03:35,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:35,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117079 s, 87.5 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:35,819::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:35,819::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:35,819::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:37,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:37,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116497 s, 87.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:38,410::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:38,410::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:38,410::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:38,410::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) (190366) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) (190366) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) (190366) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:38,411::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:03:38,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:03:38,440::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00024504 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) (190367) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:38,441::__init__::225::IOProcess::(_processLogs) (190367) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:38,442::__init__::225::IOProcess::(_processLogs) (190367) Building response >Thread-14::DEBUG::2014-08-22 09:03:38,442::__init__::225::IOProcess::(_processLogs) (190367) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:38,442::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) (190368) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) (190368) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) (190368) Building response >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) (190368) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:38,443::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) (190369) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:38,444::__init__::225::IOProcess::(_processLogs) (190369) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) (190369) Building response >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) (190369) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:38,445::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) (190370) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) (190370) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) (190370) Building response >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) (190370) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:38,446::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:03:38,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`0be111ce-a1e3-4e63-acf3-8ac08b06c9f7`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:38,717::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:03:38,717::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000191569', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00024504', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:38,717::task::1191::Storage.TaskManager.Task::(prepare) Task=`0be111ce-a1e3-4e63-acf3-8ac08b06c9f7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000191569', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00024504', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:38,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`0be111ce-a1e3-4e63-acf3-8ac08b06c9f7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:38,717::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:38,717::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:38,717::task::993::Storage.TaskManager.Task::(_decref) Task=`0be111ce-a1e3-4e63-acf3-8ac08b06c9f7`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:03:38,740::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:39,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:39,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120942 s, 84.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:39,972::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:03:39,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`76619cb3-018c-4899-a5a8-4604e47b0af6`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:39,972::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:03:39,973::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:39,973::task::1191::Storage.TaskManager.Task::(prepare) Task=`76619cb3-018c-4899-a5a8-4604e47b0af6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:39,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`76619cb3-018c-4899-a5a8-4604e47b0af6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:39,973::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:39,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:39,973::task::993::Storage.TaskManager.Task::(_decref) Task=`76619cb3-018c-4899-a5a8-4604e47b0af6`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:03:40,038::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:03:40,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d2271a7-ff6b-4ebf-9103-c1fa99fd60b6`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:03:40,039::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:03:40,039::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3adb4939-ab51-4169-8a37-5cc415b26e1c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:03:40,039::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:03:40,039::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:03:40,039::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3adb4939-ab51-4169-8a37-5cc415b26e1c`::Granted request >Thread-171::DEBUG::2014-08-22 09:03:40,040::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9d2271a7-ff6b-4ebf-9103-c1fa99fd60b6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:03:40,040::task::993::Storage.TaskManager.Task::(_decref) Task=`9d2271a7-ff6b-4ebf-9103-c1fa99fd60b6`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:03:40,042::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:40,042::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d2271a7-ff6b-4ebf-9103-c1fa99fd60b6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:40,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d2271a7-ff6b-4ebf-9103-c1fa99fd60b6`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:03:40,042::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:03:40,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:03:40,042::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:03:40,043::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:03:40,043::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:03:40,043::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:03:40,043::task::993::Storage.TaskManager.Task::(_decref) Task=`9d2271a7-ff6b-4ebf-9103-c1fa99fd60b6`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:40,825::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:40,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:40,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:41,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:41,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114524 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:03:43,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:43,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177708 s, 57.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) (190371) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) (190371) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) (190371) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:44,984::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:44,985::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:03:44,985::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:03:45,013::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000232593 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) (190372) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) (190372) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:45,014::__init__::225::IOProcess::(_processLogs) (190372) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:45,015::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:45,015::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:03:45,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:45,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012166 s, 84.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:45,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:45,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:45,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:47,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:47,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116661 s, 87.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:48,447::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:48,447::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:48,447::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:48,447::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) (190373) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) (190373) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) (190373) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:48,448::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:03:48,449::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:03:48,477::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000272264 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) (190374) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) (190374) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190374) Building response >Thread-14::DEBUG::2014-08-22 09:03:48,478::__init__::225::IOProcess::(_processLogs) (190374) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:48,479::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:48,479::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) (190375) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) (190375) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) (190375) Building response >Thread-14::DEBUG::2014-08-22 09:03:48,480::__init__::225::IOProcess::(_processLogs) (190375) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:48,481::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:48,481::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:48,481::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:48,481::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:03:48,481::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:48,481::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) (190376) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) (190376) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) (190376) Building response >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) (190376) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:48,482::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) (190377) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:48,483::__init__::225::IOProcess::(_processLogs) (190377) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:48,484::__init__::225::IOProcess::(_processLogs) (190377) Building response >Thread-14::DEBUG::2014-08-22 09:03:48,484::__init__::225::IOProcess::(_processLogs) (190377) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:48,484::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:48,484::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:03:49,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:49,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183433 s, 55.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:50,093::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:03:50,094::task::595::Storage.TaskManager.Task::(_updateState) Task=`32665184-898a-4f65-9dae-df915a41b054`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:50,095::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:03:50,097::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:50,097::task::1191::Storage.TaskManager.Task::(prepare) Task=`32665184-898a-4f65-9dae-df915a41b054`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:03:50,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`32665184-898a-4f65-9dae-df915a41b054`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:50,098::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:50,099::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:50,100::task::993::Storage.TaskManager.Task::(_decref) Task=`32665184-898a-4f65-9dae-df915a41b054`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:03:50,140::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:03:50,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`95262936-7d8d-4f3d-807f-244400085366`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:03:50,141::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:03:50,143::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8b24d185-5ce5-4643-8f8a-10a763fc1f5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:03:50,143::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:03:50,144::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:03:50,144::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8b24d185-5ce5-4643-8f8a-10a763fc1f5d`::Granted request >Thread-171::DEBUG::2014-08-22 09:03:50,145::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`95262936-7d8d-4f3d-807f-244400085366`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:03:50,146::task::993::Storage.TaskManager.Task::(_decref) Task=`95262936-7d8d-4f3d-807f-244400085366`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:03:50,155::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:50,155::task::1191::Storage.TaskManager.Task::(prepare) Task=`95262936-7d8d-4f3d-807f-244400085366`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:03:50,156::task::595::Storage.TaskManager.Task::(_updateState) Task=`95262936-7d8d-4f3d-807f-244400085366`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:03:50,156::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:03:50,157::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:03:50,157::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:03:50,158::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:03:50,158::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:03:50,159::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:03:50,160::task::993::Storage.TaskManager.Task::(_decref) Task=`95262936-7d8d-4f3d-807f-244400085366`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:50,838::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:50,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:50,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:51,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:51,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117721 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:03:53,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:53,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124201 s, 82.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:03:54,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`5870fa76-b37c-4541-93fa-c66db9354487`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:03:54,230::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:03:54,230::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232593', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000272264', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:54,230::task::1191::Storage.TaskManager.Task::(prepare) Task=`5870fa76-b37c-4541-93fa-c66db9354487`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232593', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000272264', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:03:54,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`5870fa76-b37c-4541-93fa-c66db9354487`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:03:54,230::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:03:54,230::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:03:54,231::task::993::Storage.TaskManager.Task::(_decref) Task=`5870fa76-b37c-4541-93fa-c66db9354487`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:03:54,253::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:55,016::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:55,017::__init__::225::IOProcess::(_processLogs) (190378) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:55,017::__init__::225::IOProcess::(_processLogs) (190378) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:55,017::__init__::225::IOProcess::(_processLogs) (190378) Building response >Thread-14::DEBUG::2014-08-22 09:03:55,017::__init__::225::IOProcess::(_processLogs) (190378) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:55,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:55,017::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:03:55,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:03:55,045::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000317199 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:55,046::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:03:55,046::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:55,046::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:55,046::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:55,046::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) (190379) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) (190379) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) DEBUG|(190379) Building response >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) (190379) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:55,047::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:55,845::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:55,846::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:03:55,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:03:55,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:56,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121279 s, 84.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:03:58,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:03:58,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0126646 s, 80.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:58,485::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:58,486::__init__::225::IOProcess::(_processLogs) (190380) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:58,486::__init__::225::IOProcess::(_processLogs) (190380) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:58,486::__init__::225::IOProcess::(_processLogs) (190380) Building response >Thread-14::DEBUG::2014-08-22 09:03:58,486::__init__::225::IOProcess::(_processLogs) (190380) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:58,486::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:58,486::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:03:58,487::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:03:58,515::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000217975 s, 2.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:03:58,515::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:03:58,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) (190381) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) (190381) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) (190381) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:58,516::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:03:58,517::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:03:58,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:58,517::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:58,517::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:58,517::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) (190382) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) (190382) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) (190382) Building response >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) (190382) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:58,518::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) (190383) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) (190383) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) (190383) Building response >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) (190383) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:58,519::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:03:58,520::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:03:58,521::__init__::225::IOProcess::(_processLogs) (190384) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:03:58,521::__init__::225::IOProcess::(_processLogs) (190384) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:03:58,521::__init__::225::IOProcess::(_processLogs) (190384) Building response >Thread-14::DEBUG::2014-08-22 09:03:58,521::__init__::225::IOProcess::(_processLogs) (190384) Queuing response >Thread-14::DEBUG::2014-08-22 09:03:58,521::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:03:58,521::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:04:00,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:00,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121173 s, 84.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:00,204::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-13::DEBUG::2014-08-22 09:04:00,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`a21862ed-a3f2-4851-929c-3040033b67c3`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:00,204::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:04:00,205::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:00,205::task::1191::Storage.TaskManager.Task::(prepare) Task=`a21862ed-a3f2-4851-929c-3040033b67c3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:00,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`a21862ed-a3f2-4851-929c-3040033b67c3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:00,205::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:00,205::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:00,205::task::993::Storage.TaskManager.Task::(_decref) Task=`a21862ed-a3f2-4851-929c-3040033b67c3`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:04:00,238::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-171::DEBUG::2014-08-22 09:04:00,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbd0513d-eedd-432d-a36a-96322e3bf0bb`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:04:00,238::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:04:00,239::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bec4ff56-bad4-4d28-a1cb-e8eb573a4bff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:04:00,239::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:04:00,239::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:04:00,239::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bec4ff56-bad4-4d28-a1cb-e8eb573a4bff`::Granted request >Thread-171::DEBUG::2014-08-22 09:04:00,239::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dbd0513d-eedd-432d-a36a-96322e3bf0bb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:04:00,239::task::993::Storage.TaskManager.Task::(_decref) Task=`dbd0513d-eedd-432d-a36a-96322e3bf0bb`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:04:00,242::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:00,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`dbd0513d-eedd-432d-a36a-96322e3bf0bb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:00,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbd0513d-eedd-432d-a36a-96322e3bf0bb`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:04:00,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:04:00,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:04:00,242::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:04:00,242::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:04:00,242::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:04:00,242::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:04:00,242::task::993::Storage.TaskManager.Task::(_decref) Task=`dbd0513d-eedd-432d-a36a-96322e3bf0bb`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:00,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:00,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:00,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:02,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:02,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181948 s, 56.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:04,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:04,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119284 s, 85.8 MB/s\n'; <rc> = 0 >Thread-91::DEBUG::2014-08-22 09:04:05,048::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,050::__init__::225::IOProcess::(_processLogs) (190385) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:04:05,051::__init__::225::IOProcess::(_processLogs) (190385) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:04:05,051::__init__::225::IOProcess::(_processLogs) (190385) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,051::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,051::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-14::DEBUG::2014-08-22 09:04:05,051::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:05,051::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,052::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:05,053::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,054::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,055::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,055::__init__::225::IOProcess::(_processLogs) (190386) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:04:05,056::__init__::225::IOProcess::(_processLogs) (190386) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:04:05,057::__init__::225::IOProcess::(_processLogs) (190386) Building response >Thread-14::DEBUG::2014-08-22 09:04:05,057::__init__::225::IOProcess::(_processLogs) (190386) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,058::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,058::__init__::225::IOProcess::(_processLogs) Sending response sized 195 >Thread-91::DEBUG::2014-08-22 09:04:05,060::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) (190387) Finding callback 'touch'... >Thread-14::DEBUG::2014-08-22 09:04:05,061::__init__::225::IOProcess::(_processLogs) (190387) Got request for method 'touch' >Thread-14::DEBUG::2014-08-22 09:04:05,089::__init__::225::IOProcess::(_processLogs) (190387) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,090::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-91::DEBUG::2014-08-22 09:04:05,090::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-22 09:04:05,090::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-14::DEBUG::2014-08-22 09:04:05,090::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,090::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,090::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,090::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,091::__init__::225::IOProcess::(_processLogs) (190388) Finding callback 'readfile'... >Thread-14::DEBUG::2014-08-22 09:04:05,091::__init__::225::IOProcess::(_processLogs) (190388) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-22 09:04:05,091::__init__::225::IOProcess::(_processLogs) (190388) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,091::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,091::__init__::225::IOProcess::(_processLogs) Sending response sized 606 >Thread-91::DEBUG::2014-08-22 09:04:05,091::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) (190389) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) (190389) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) (190389) Building response >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) (190389) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,092::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) (190390) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:04:05,093::__init__::225::IOProcess::(_processLogs) (190390) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:04:05,094::__init__::225::IOProcess::(_processLogs) (190390) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,094::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,094::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-91::DEBUG::2014-08-22 09:04:05,094::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-91::INFO::2014-08-22 09:04:05,094::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-91::INFO::2014-08-22 09:04:05,094::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) (190391) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) (190391) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) (190391) Building response >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) (190391) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,095::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,096::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:04:05,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:04:05,124::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000269636 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) (190392) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:05,125::__init__::225::IOProcess::(_processLogs) (190392) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:05,126::__init__::225::IOProcess::(_processLogs) (190392) Building response >Thread-14::DEBUG::2014-08-22 09:04:05,126::__init__::225::IOProcess::(_processLogs) (190392) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:05,126::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:05,126::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:05,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:05,861::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:05,862::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:06,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:06,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116939 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:08,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:08,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116038 s, 88.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:08,522::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:08,522::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:08,522::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:08,522::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) (190393) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) (190393) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) DEBUG|(190393) Building response >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) (190393) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:08,523::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:04:08,524::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:04:08,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000228318 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:08,553::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:08,554::__init__::225::IOProcess::(_processLogs) (190394) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:08,554::__init__::225::IOProcess::(_processLogs) (190394) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:08,554::__init__::225::IOProcess::(_processLogs) (190394) Building response >Thread-14::DEBUG::2014-08-22 09:04:08,554::__init__::225::IOProcess::(_processLogs) (190394) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:08,554::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:08,554::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:08,555::__init__::225::IOProcess::(_processLogs) (190395) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) (190395) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) (190395) Building response >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) (190395) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:08,556::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) (190396) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) (190396) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) (190396) Building response >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) (190396) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:08,557::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:08,558::__init__::225::IOProcess::(_processLogs) (190397) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:08,559::__init__::225::IOProcess::(_processLogs) (190397) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:08,559::__init__::225::IOProcess::(_processLogs) (190397) Building response >Thread-14::DEBUG::2014-08-22 09:04:08,559::__init__::225::IOProcess::(_processLogs) (190397) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:08,559::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:08,559::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:04:09,502::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e4dab87-d6f4-4bb0-865d-b79bdc29216c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:09,502::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:04:09,502::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000269636', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000228318', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:09,503::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e4dab87-d6f4-4bb0-865d-b79bdc29216c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000269636', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000228318', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:09,503::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e4dab87-d6f4-4bb0-865d-b79bdc29216c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:09,503::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:09,503::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:09,503::task::993::Storage.TaskManager.Task::(_decref) Task=`0e4dab87-d6f4-4bb0-865d-b79bdc29216c`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:04:09,529::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:04:10,475::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:04:10,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`b50a3ed5-cc00-42e8-b13b-2461610c042d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:10,475::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:04:10,476::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:10,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`b50a3ed5-cc00-42e8-b13b-2461610c042d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:10,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`b50a3ed5-cc00-42e8-b13b-2461610c042d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:10,476::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:10,476::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:10,476::task::993::Storage.TaskManager.Task::(_decref) Task=`b50a3ed5-cc00-42e8-b13b-2461610c042d`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:04:10,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-171::DEBUG::2014-08-22 09:04:10,522::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:04:10,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`80be09a6-51a7-4204-b1c5-d5ba4426119b`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:04:10,523::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:04:10,523::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4bead405-5292-4c1e-b3fb-5def7528d691`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:04:10,523::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:04:10,523::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:04:10,523::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4bead405-5292-4c1e-b3fb-5def7528d691`::Granted request >Thread-171::DEBUG::2014-08-22 09:04:10,524::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`80be09a6-51a7-4204-b1c5-d5ba4426119b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:04:10,524::task::993::Storage.TaskManager.Task::(_decref) Task=`80be09a6-51a7-4204-b1c5-d5ba4426119b`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:04:10,526::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:10,526::task::1191::Storage.TaskManager.Task::(prepare) Task=`80be09a6-51a7-4204-b1c5-d5ba4426119b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:10,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`80be09a6-51a7-4204-b1c5-d5ba4426119b`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:04:10,526::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:04:10,526::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:04:10,527::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:04:10,527::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:04:10,527::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:04:10,527::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:04:10,527::task::993::Storage.TaskManager.Task::(_decref) Task=`80be09a6-51a7-4204-b1c5-d5ba4426119b`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:04:10,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120665 s, 84.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:10,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:10,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:10,869::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-183::DEBUG::2014-08-22 09:04:12,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`99d5b355-df81-4ad7-a034-dcabbec75f9f`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:04:12,571::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:12,572::__init__::225::IOProcess::(_processLogs) (190398) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:04:12,573::__init__::225::IOProcess::(_processLogs) (190398) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:04:12,573::__init__::225::IOProcess::(_processLogs) DEBUG|(190398) Building response >Thread-14::DEBUG::2014-08-22 09:04:12,573::__init__::225::IOProcess::(_processLogs) (190398) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:12,573::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:12,573::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) (190399) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) (190399) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:04:12,574::__init__::225::IOProcess::(_processLogs) (190399) Building response >Thread-14::DEBUG::2014-08-22 09:04:12,575::__init__::225::IOProcess::(_processLogs) (190399) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:12,575::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:12,575::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:04:12,575::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:04:12,575::task::1191::Storage.TaskManager.Task::(prepare) Task=`99d5b355-df81-4ad7-a034-dcabbec75f9f`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:04:12,575::task::595::Storage.TaskManager.Task::(_updateState) Task=`99d5b355-df81-4ad7-a034-dcabbec75f9f`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:04:12,575::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:04:12,575::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:04:12,576::task::993::Storage.TaskManager.Task::(_decref) Task=`99d5b355-df81-4ad7-a034-dcabbec75f9f`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:04:12,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:12,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115734 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:14,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:14,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195657 s, 52.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:15,127::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:15,127::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:15,127::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:15,127::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) (190400) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) (190400) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) (190400) Building response >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) (190400) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:15,128::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:04:15,129::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:04:15,157::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000233069 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) (190401) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:15,158::__init__::225::IOProcess::(_processLogs) (190401) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:15,159::__init__::225::IOProcess::(_processLogs) (190401) Building response >Thread-14::DEBUG::2014-08-22 09:04:15,159::__init__::225::IOProcess::(_processLogs) (190401) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:15,159::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:15,159::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:15,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:15,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:15,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:16,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:16,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123701 s, 82.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:18,560::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:18,560::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:18,560::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:18,560::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) (190402) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) (190402) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) (190402) Building response >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) (190402) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:18,561::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:04:18,562::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:04:18,599::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00912107 s, 62.5 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:18,599::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:18,599::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) (190403) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) (190403) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) (190403) Building response >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) (190403) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:18,600::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:04:18,601::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:18,601::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:04:18,601::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) (190404) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) (190404) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) (190404) Building response >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) (190404) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:18,602::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:18,603::__init__::225::IOProcess::(_processLogs) (190405) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) (190405) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) (190405) Building response >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) (190405) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:18,604::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) (190406) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) (190406) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) (190406) Building response >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) (190406) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:18,605::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:04:18,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:18,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120092 s, 85.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:20,602::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:04:20,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`01e98c7f-a0df-4f6f-9568-2179fdc57657`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:20,603::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:04:20,603::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:20,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`01e98c7f-a0df-4f6f-9568-2179fdc57657`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:20,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`01e98c7f-a0df-4f6f-9568-2179fdc57657`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:20,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:20,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:20,603::task::993::Storage.TaskManager.Task::(_decref) Task=`01e98c7f-a0df-4f6f-9568-2179fdc57657`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:04:20,637::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:04:20,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3b54a88-011f-48f9-b579-36175e11e9e7`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:04:20,638::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:04:20,638::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c2a7efe9-0517-455b-9acd-3789a29a89ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:04:20,638::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:04:20,638::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:04:20,638::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c2a7efe9-0517-455b-9acd-3789a29a89ab`::Granted request >Thread-171::DEBUG::2014-08-22 09:04:20,639::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a3b54a88-011f-48f9-b579-36175e11e9e7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:04:20,639::task::993::Storage.TaskManager.Task::(_decref) Task=`a3b54a88-011f-48f9-b579-36175e11e9e7`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:04:20,641::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:20,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3b54a88-011f-48f9-b579-36175e11e9e7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:20,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3b54a88-011f-48f9-b579-36175e11e9e7`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:04:20,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:04:20,641::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:04:20,641::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:04:20,641::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:04:20,641::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:04:20,641::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:04:20,642::task::993::Storage.TaskManager.Task::(_decref) Task=`a3b54a88-011f-48f9-b579-36175e11e9e7`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:20,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:20,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:20,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:20,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:21,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117621 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:23,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:23,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115099 s, 89.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:24,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`47dbf8f3-9661-45b7-94a7-fdde3d1047ba`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:24,694::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:04:24,694::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233069', 'lastCheck': '9.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00912107', 'lastCheck': '6.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:24,694::task::1191::Storage.TaskManager.Task::(prepare) Task=`47dbf8f3-9661-45b7-94a7-fdde3d1047ba`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233069', 'lastCheck': '9.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00912107', 'lastCheck': '6.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:24,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`47dbf8f3-9661-45b7-94a7-fdde3d1047ba`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:24,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:24,694::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:24,694::task::993::Storage.TaskManager.Task::(_decref) Task=`47dbf8f3-9661-45b7-94a7-fdde3d1047ba`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:04:24,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:25,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:25,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118868 s, 86.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:25,164::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:25,164::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:25,165::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:25,166::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:25,166::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:25,167::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:25,167::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:25,168::__init__::225::IOProcess::(_processLogs) (190407) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:25,168::__init__::225::IOProcess::(_processLogs) (190407) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:25,169::__init__::225::IOProcess::(_processLogs) DEBUG|(190407) Building response >Thread-14::DEBUG::2014-08-22 09:04:25,170::__init__::225::IOProcess::(_processLogs) (190407) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:25,170::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:25,171::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:04:25,174::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:04:25,203::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000242644 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:25,205::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:25,206::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:25,206::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:25,207::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:25,208::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:25,208::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:25,209::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:25,210::__init__::225::IOProcess::(_processLogs) (190408) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:25,210::__init__::225::IOProcess::(_processLogs) (190408) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:25,211::__init__::225::IOProcess::(_processLogs) (190408) Building response >Thread-14::DEBUG::2014-08-22 09:04:25,212::__init__::225::IOProcess::(_processLogs) (190408) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:25,212::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:25,213::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:25,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:25,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:25,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:27,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:27,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119821 s, 85.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:28,607::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:28,607::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) (190409) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) (190409) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) (190409) Building response >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) (190409) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:28,608::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:04:28,609::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:04:28,637::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234419 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) (190410) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) (190410) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:28,638::__init__::225::IOProcess::(_processLogs) DEBUG|(190410) Building response >Thread-14::DEBUG::2014-08-22 09:04:28,639::__init__::225::IOProcess::(_processLogs) (190410) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:28,639::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:28,639::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:04:28,639::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) (190411) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) (190411) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) (190411) Building response >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) (190411) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:28,640::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:28,641::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) (190412) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) (190412) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) (190412) Building response >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) (190412) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:28,642::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) (190413) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) (190413) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) (190413) Building response >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) (190413) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:28,643::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:28,644::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:04:29,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:29,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119649 s, 85.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:30,690::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:04:30,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`872d7da9-0717-4945-9e8e-512d84c88136`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:30,691::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:04:30,692::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:30,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`872d7da9-0717-4945-9e8e-512d84c88136`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:30,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`872d7da9-0717-4945-9e8e-512d84c88136`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:30,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:30,692::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:30,692::task::993::Storage.TaskManager.Task::(_decref) Task=`872d7da9-0717-4945-9e8e-512d84c88136`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:04:30,726::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:04:30,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f41962a-e714-44e6-9083-537e47a513d1`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:04:30,727::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:04:30,727::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`83837d8e-166c-4ba5-a1b2-c292a57e7e38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:04:30,727::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:04:30,727::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:04:30,727::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`83837d8e-166c-4ba5-a1b2-c292a57e7e38`::Granted request >Thread-171::DEBUG::2014-08-22 09:04:30,728::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3f41962a-e714-44e6-9083-537e47a513d1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:04:30,728::task::993::Storage.TaskManager.Task::(_decref) Task=`3f41962a-e714-44e6-9083-537e47a513d1`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:04:30,730::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:30,730::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f41962a-e714-44e6-9083-537e47a513d1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:30,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f41962a-e714-44e6-9083-537e47a513d1`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:04:30,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:04:30,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:04:30,730::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:04:30,730::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:04:30,730::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:04:30,731::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:04:30,731::task::993::Storage.TaskManager.Task::(_decref) Task=`3f41962a-e714-44e6-9083-537e47a513d1`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:30,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:30,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:30,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:31,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:31,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116056 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:33,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:33,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117632 s, 87.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:35,217::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:35,217::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:35,217::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) (190414) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) (190414) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) DEBUG|(190414) Building response >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) (190414) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:35,218::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:04:35,219::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:04:35,247::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000287094 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) (190415) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) (190415) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) (190415) Building response >Thread-14::DEBUG::2014-08-22 09:04:35,248::__init__::225::IOProcess::(_processLogs) (190415) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:35,249::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:35,249::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:04:35,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:35,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117853 s, 86.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:35,903::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:35,903::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:35,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:37,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:37,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116752 s, 87.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) (190416) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:38,645::__init__::225::IOProcess::(_processLogs) (190416) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:38,646::__init__::225::IOProcess::(_processLogs) (190416) Building response >Thread-14::DEBUG::2014-08-22 09:04:38,646::__init__::225::IOProcess::(_processLogs) (190416) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:38,646::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:38,646::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:04:38,646::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:04:38,674::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000278002 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:38,675::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:38,675::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:38,675::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:38,675::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) (190417) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) (190417) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) (190417) Building response >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) (190417) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:38,676::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:04:38,677::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:38,677::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:04:38,677::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:38,677::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:38,677::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:38,677::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) (190418) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) (190418) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) (190418) Building response >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) (190418) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:38,678::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) (190419) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) (190419) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) (190419) Building response >Thread-14::DEBUG::2014-08-22 09:04:38,679::__init__::225::IOProcess::(_processLogs) (190419) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:38,680::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) (190420) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) (190420) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) (190420) Building response >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) (190420) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:38,681::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:04:39,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:39,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122213 s, 83.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:39,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`c26cda0e-1735-4275-a7ed-9ee52f9f53ab`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:39,963::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:04:39,963::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000287094', 'lastCheck': '4.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000278002', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:39,964::task::1191::Storage.TaskManager.Task::(prepare) Task=`c26cda0e-1735-4275-a7ed-9ee52f9f53ab`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000287094', 'lastCheck': '4.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000278002', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:39,964::task::595::Storage.TaskManager.Task::(_updateState) Task=`c26cda0e-1735-4275-a7ed-9ee52f9f53ab`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:39,964::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:39,964::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:39,964::task::993::Storage.TaskManager.Task::(_decref) Task=`c26cda0e-1735-4275-a7ed-9ee52f9f53ab`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:04:39,987::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:04:40,778::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:04:40,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`05c6327d-31b9-4e45-9653-adfa30055b9a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:40,778::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:04:40,778::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:40,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`05c6327d-31b9-4e45-9653-adfa30055b9a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:40,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`05c6327d-31b9-4e45-9653-adfa30055b9a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:40,779::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:40,779::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:40,779::task::993::Storage.TaskManager.Task::(_decref) Task=`05c6327d-31b9-4e45-9653-adfa30055b9a`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:04:40,812::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:04:40,812::task::595::Storage.TaskManager.Task::(_updateState) Task=`11ec593e-8fc3-459c-84a2-5b118816d3f5`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:04:40,812::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:04:40,813::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12f4ab93-d130-4b9b-ab28-19563f3acef9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:04:40,813::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:04:40,813::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:04:40,813::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12f4ab93-d130-4b9b-ab28-19563f3acef9`::Granted request >Thread-171::DEBUG::2014-08-22 09:04:40,813::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`11ec593e-8fc3-459c-84a2-5b118816d3f5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:04:40,813::task::993::Storage.TaskManager.Task::(_decref) Task=`11ec593e-8fc3-459c-84a2-5b118816d3f5`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:04:40,815::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:40,815::task::1191::Storage.TaskManager.Task::(prepare) Task=`11ec593e-8fc3-459c-84a2-5b118816d3f5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:40,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`11ec593e-8fc3-459c-84a2-5b118816d3f5`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:04:40,816::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:04:40,816::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:04:40,816::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:04:40,816::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:04:40,816::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:04:40,816::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:04:40,816::task::993::Storage.TaskManager.Task::(_decref) Task=`11ec593e-8fc3-459c-84a2-5b118816d3f5`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:40,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:40,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:40,912::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:41,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:41,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161349 s, 63.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:43,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:43,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117721 s, 87.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:45,250::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:45,251::__init__::225::IOProcess::(_processLogs) (190421) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:45,251::__init__::225::IOProcess::(_processLogs) (190421) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:45,251::__init__::225::IOProcess::(_processLogs) (190421) Building response >Thread-14::DEBUG::2014-08-22 09:04:45,251::__init__::225::IOProcess::(_processLogs) (190421) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:45,251::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:45,251::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:04:45,252::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:04:45,280::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000231191 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:45,280::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:45,280::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) (190422) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) (190422) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) (190422) Building response >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) (190422) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:45,281::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:45,282::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:04:45,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:45,917::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:45,918::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:45,918::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:45,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121927 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:47,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:48,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116874 s, 87.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:48,682::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:48,682::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:48,682::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) (190423) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) (190423) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) (190423) Building response >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) (190423) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:48,683::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:04:48,684::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:04:48,712::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000245874 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) (190424) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:48,713::__init__::225::IOProcess::(_processLogs) (190424) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:48,714::__init__::225::IOProcess::(_processLogs) (190424) Building response >Thread-14::DEBUG::2014-08-22 09:04:48,714::__init__::225::IOProcess::(_processLogs) (190424) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:48,714::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:48,714::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) (190425) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:48,715::__init__::225::IOProcess::(_processLogs) (190425) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) (190425) Building response >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) (190425) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:04:48,716::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) (190426) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) (190426) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) (190426) Building response >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) (190426) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:48,717::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:48,718::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:48,719::__init__::225::IOProcess::(_processLogs) (190427) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:48,719::__init__::225::IOProcess::(_processLogs) (190427) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:48,719::__init__::225::IOProcess::(_processLogs) DEBUG|(190427) Building response >Thread-14::DEBUG::2014-08-22 09:04:48,719::__init__::225::IOProcess::(_processLogs) (190427) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:48,719::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:48,719::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:04:50,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:50,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227993 s, 44.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:50,865::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:04:50,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`84f6f264-84fd-4ae4-8e1d-3d61d60330c1`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:50,866::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:04:50,866::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:50,866::task::1191::Storage.TaskManager.Task::(prepare) Task=`84f6f264-84fd-4ae4-8e1d-3d61d60330c1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:04:50,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`84f6f264-84fd-4ae4-8e1d-3d61d60330c1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:50,866::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:50,866::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:50,866::task::993::Storage.TaskManager.Task::(_decref) Task=`84f6f264-84fd-4ae4-8e1d-3d61d60330c1`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:04:50,899::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:04:50,899::task::595::Storage.TaskManager.Task::(_updateState) Task=`90fa5ae9-f703-46e9-97e4-35f2e81cabd2`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:04:50,899::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:04:50,899::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`506b410c-d484-49d6-989b-b80fef0184f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:04:50,900::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:04:50,900::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:04:50,900::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`506b410c-d484-49d6-989b-b80fef0184f4`::Granted request >Thread-171::DEBUG::2014-08-22 09:04:50,900::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`90fa5ae9-f703-46e9-97e4-35f2e81cabd2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:04:50,900::task::993::Storage.TaskManager.Task::(_decref) Task=`90fa5ae9-f703-46e9-97e4-35f2e81cabd2`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:04:50,902::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:50,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`90fa5ae9-f703-46e9-97e4-35f2e81cabd2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:04:50,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`90fa5ae9-f703-46e9-97e4-35f2e81cabd2`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:04:50,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:04:50,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:04:50,903::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:04:50,903::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:04:50,903::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:04:50,903::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:04:50,903::task::993::Storage.TaskManager.Task::(_decref) Task=`90fa5ae9-f703-46e9-97e4-35f2e81cabd2`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:50,924::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:50,925::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:50,925::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:52,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:52,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0149662 s, 68.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:54,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:54,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115915 s, 88.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:04:55,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a191c97-310b-4026-99b4-6b735d958717`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:04:55,230::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:04:55,230::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000231191', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245874', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:55,230::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a191c97-310b-4026-99b4-6b735d958717`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000231191', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245874', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:04:55,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a191c97-310b-4026-99b4-6b735d958717`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:04:55,231::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:04:55,231::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:04:55,231::task::993::Storage.TaskManager.Task::(_decref) Task=`1a191c97-310b-4026-99b4-6b735d958717`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:04:55,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:55,283::__init__::225::IOProcess::(_processLogs) (190428) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:55,284::__init__::225::IOProcess::(_processLogs) (190428) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:55,284::__init__::225::IOProcess::(_processLogs) (190428) Building response >Thread-14::DEBUG::2014-08-22 09:04:55,284::__init__::225::IOProcess::(_processLogs) (190428) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:55,284::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:55,284::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:04:55,284::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:04:55,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000321451 s, 1.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:55,313::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:55,313::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:04:55,313::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:55,313::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:55,313::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) (190429) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) (190429) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) (190429) Building response >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) (190429) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:55,314::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:55,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:55,932::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:04:55,932::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:04:56,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:56,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118717 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:04:58,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:04:58,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116811 s, 87.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:58,720::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:58,720::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:58,720::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:58,720::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:58,720::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) (190430) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) (190430) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) (190430) Building response >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) (190430) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:58,721::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:04:58,722::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:04:58,750::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000232409 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) (190431) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:04:58,751::__init__::225::IOProcess::(_processLogs) (190431) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:04:58,752::__init__::225::IOProcess::(_processLogs) (190431) Building response >Thread-14::DEBUG::2014-08-22 09:04:58,752::__init__::225::IOProcess::(_processLogs) (190431) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:58,752::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:58,752::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) (190432) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) (190432) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:58,753::__init__::225::IOProcess::(_processLogs) (190432) Building response >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) (190432) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:58,754::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) (190433) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) (190433) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) (190433) Building response >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) (190433) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:58,755::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:04:58,756::__init__::225::IOProcess::(_processLogs) (190434) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:04:58,757::__init__::225::IOProcess::(_processLogs) (190434) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:04:58,757::__init__::225::IOProcess::(_processLogs) (190434) Building response >Thread-14::DEBUG::2014-08-22 09:04:58,757::__init__::225::IOProcess::(_processLogs) (190434) Queuing response >Thread-14::DEBUG::2014-08-22 09:04:58,757::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:04:58,757::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:05:00,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:00,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117417 s, 87.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:00,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:00,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:00,939::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:00,952::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:05:00,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`c981c735-ba29-4f1d-bb75-0cb3116a1899`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:00,952::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:05:00,953::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:00,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`c981c735-ba29-4f1d-bb75-0cb3116a1899`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:00,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`c981c735-ba29-4f1d-bb75-0cb3116a1899`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:00,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:00,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:00,953::task::993::Storage.TaskManager.Task::(_decref) Task=`c981c735-ba29-4f1d-bb75-0cb3116a1899`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:05:00,985::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:05:00,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd7e0b70-d63f-42a9-928b-7658c72d7288`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:05:00,986::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:05:00,986::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`604c2b81-39fd-4571-9d57-d2c93fdf81bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:05:00,986::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:05:00,986::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:05:00,986::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`604c2b81-39fd-4571-9d57-d2c93fdf81bc`::Granted request >Thread-171::DEBUG::2014-08-22 09:05:00,987::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cd7e0b70-d63f-42a9-928b-7658c72d7288`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:05:00,987::task::993::Storage.TaskManager.Task::(_decref) Task=`cd7e0b70-d63f-42a9-928b-7658c72d7288`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:05:00,989::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:00,989::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd7e0b70-d63f-42a9-928b-7658c72d7288`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:00,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd7e0b70-d63f-42a9-928b-7658c72d7288`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:05:00,989::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:05:00,989::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:05:00,989::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:05:00,989::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:05:00,989::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:05:00,989::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:05:00,990::task::993::Storage.TaskManager.Task::(_decref) Task=`cd7e0b70-d63f-42a9-928b-7658c72d7288`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:05:02,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:02,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116627 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:04,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:04,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114381 s, 89.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:05,315::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) (190435) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) (190435) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190435) Building response >Thread-14::DEBUG::2014-08-22 09:05:05,316::__init__::225::IOProcess::(_processLogs) (190435) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:05,317::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:05,317::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:05:05,317::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:05:05,345::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000252451 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:05,346::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:05,346::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:05,346::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:05,346::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:05,346::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:05,346::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) (190436) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) (190436) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) (190436) Building response >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) (190436) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:05,347::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:05,945::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:05,945::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:05,946::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:06,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:06,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120733 s, 84.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:08,758::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:08,758::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:08,758::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:08,758::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) (190437) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) (190437) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) (190437) Building response >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) (190437) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:08,759::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:05:08,760::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:05:08,788::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000236829 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) (190438) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) (190438) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:08,789::__init__::225::IOProcess::(_processLogs) (190438) Building response >Thread-14::DEBUG::2014-08-22 09:05:08,790::__init__::225::IOProcess::(_processLogs) (190438) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:08,790::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:08,790::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:05:08,790::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) (190439) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) (190439) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) (190439) Building response >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) (190439) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:08,791::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:08,792::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) (190440) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) (190440) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) (190440) Building response >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) (190440) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:08,793::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) (190441) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) (190441) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) (190441) Building response >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) (190441) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:08,794::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:08,795::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:05:08,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:08,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119931 s, 85.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:05:10,432::task::595::Storage.TaskManager.Task::(_updateState) Task=`b82bbafa-f02f-4cde-9507-a4707b22c0c6`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:10,432::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:05:10,432::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252451', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000236829', 'lastCheck': '1.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:10,432::task::1191::Storage.TaskManager.Task::(prepare) Task=`b82bbafa-f02f-4cde-9507-a4707b22c0c6`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252451', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000236829', 'lastCheck': '1.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:10,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`b82bbafa-f02f-4cde-9507-a4707b22c0c6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:10,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:10,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:10,433::task::993::Storage.TaskManager.Task::(_decref) Task=`b82bbafa-f02f-4cde-9507-a4707b22c0c6`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:05:10,458::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:10,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:10,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011883 s, 86.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:10,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:10,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:10,970::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:11,056::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:05:11,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`46ae6900-e1d5-4cea-bb95-f7acd3c73455`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:11,057::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:05:11,057::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:11,057::task::1191::Storage.TaskManager.Task::(prepare) Task=`46ae6900-e1d5-4cea-bb95-f7acd3c73455`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:11,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`46ae6900-e1d5-4cea-bb95-f7acd3c73455`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:11,057::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:11,057::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:11,057::task::993::Storage.TaskManager.Task::(_decref) Task=`46ae6900-e1d5-4cea-bb95-f7acd3c73455`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:05:11,081::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:05:11,081::task::595::Storage.TaskManager.Task::(_updateState) Task=`480957e8-155f-4129-9f32-1bf5c0a9a573`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:05:11,081::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:05:11,082::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b554701d-d608-4d09-933e-174971098dc9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:05:11,082::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:05:11,082::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:05:11,082::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b554701d-d608-4d09-933e-174971098dc9`::Granted request >Thread-171::DEBUG::2014-08-22 09:05:11,082::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`480957e8-155f-4129-9f32-1bf5c0a9a573`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:05:11,082::task::993::Storage.TaskManager.Task::(_decref) Task=`480957e8-155f-4129-9f32-1bf5c0a9a573`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:05:11,084::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:11,084::task::1191::Storage.TaskManager.Task::(prepare) Task=`480957e8-155f-4129-9f32-1bf5c0a9a573`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:11,085::task::595::Storage.TaskManager.Task::(_updateState) Task=`480957e8-155f-4129-9f32-1bf5c0a9a573`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:05:11,085::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:05:11,085::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:05:11,085::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:05:11,085::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:05:11,085::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:05:11,085::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:05:11,085::task::993::Storage.TaskManager.Task::(_decref) Task=`480957e8-155f-4129-9f32-1bf5c0a9a573`::ref 0 aborting False >Thread-183::DEBUG::2014-08-22 09:05:12,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`24402c3a-b5e9-4f2e-b17b-f799be00080a`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:05:12,638::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) (190442) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) (190442) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:05:12,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190442) Building response >Thread-14::DEBUG::2014-08-22 09:05:12,640::__init__::225::IOProcess::(_processLogs) (190442) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:12,640::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:05:12,640::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:05:12,640::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:12,640::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) (190443) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) (190443) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) (190443) Building response >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) (190443) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:12,641::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:05:12,642::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:05:12,642::task::1191::Storage.TaskManager.Task::(prepare) Task=`24402c3a-b5e9-4f2e-b17b-f799be00080a`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:05:12,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`24402c3a-b5e9-4f2e-b17b-f799be00080a`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:05:12,642::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:05:12,642::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:05:12,642::task::993::Storage.TaskManager.Task::(_decref) Task=`24402c3a-b5e9-4f2e-b17b-f799be00080a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:05:12,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:13,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117997 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:15,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:15,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117149 s, 87.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:15,348::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:15,348::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) (190444) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) (190444) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) (190444) Building response >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) (190444) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:15,349::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:05:15,350::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:05:15,378::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219241 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:15,379::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:15,379::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:15,379::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:15,379::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) (190445) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) (190445) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) (190445) Building response >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) (190445) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:15,380::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:15,977::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:15,977::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:15,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:17,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:17,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118879 s, 86.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:18,796::__init__::225::IOProcess::(_processLogs) (190446) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:18,797::__init__::225::IOProcess::(_processLogs) (190446) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:18,797::__init__::225::IOProcess::(_processLogs) (190446) Building response >Thread-14::DEBUG::2014-08-22 09:05:18,797::__init__::225::IOProcess::(_processLogs) (190446) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:18,797::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:18,797::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:05:18,798::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:05:18,826::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000224242 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:18,826::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) (190447) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) (190447) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) (190447) Building response >Thread-14::DEBUG::2014-08-22 09:05:18,827::__init__::225::IOProcess::(_processLogs) (190447) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:18,828::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:18,828::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:05:18,828::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) (190448) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) (190448) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) (190448) Building response >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) (190448) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:18,829::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:18,830::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:18,830::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:18,830::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:05:18,830::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:18,830::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:18,830::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) (190449) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) (190449) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) (190449) Building response >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) (190449) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:18,831::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) (190450) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) (190450) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:18,832::__init__::225::IOProcess::(_processLogs) (190450) Building response >Thread-14::DEBUG::2014-08-22 09:05:18,833::__init__::225::IOProcess::(_processLogs) (190450) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:18,833::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:18,833::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:05:19,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:19,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136855 s, 74.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:20,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:20,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:20,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:21,133::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:05:21,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`5701c72f-c38e-4a20-be81-db467613ce41`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:21,134::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:05:21,134::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:21,134::task::1191::Storage.TaskManager.Task::(prepare) Task=`5701c72f-c38e-4a20-be81-db467613ce41`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:21,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`5701c72f-c38e-4a20-be81-db467613ce41`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:21,134::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:21,135::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:21,135::task::993::Storage.TaskManager.Task::(_decref) Task=`5701c72f-c38e-4a20-be81-db467613ce41`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:05:21,169::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:05:21,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`092e0c99-2c2c-482c-ba2a-59ef644afcc9`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:05:21,170::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:05:21,170::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e0b6bb69-8606-43fe-a265-0069a9aa2139`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:05:21,170::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:05:21,170::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:05:21,170::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e0b6bb69-8606-43fe-a265-0069a9aa2139`::Granted request >Thread-171::DEBUG::2014-08-22 09:05:21,171::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`092e0c99-2c2c-482c-ba2a-59ef644afcc9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:05:21,171::task::993::Storage.TaskManager.Task::(_decref) Task=`092e0c99-2c2c-482c-ba2a-59ef644afcc9`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:05:21,173::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:21,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`092e0c99-2c2c-482c-ba2a-59ef644afcc9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:21,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`092e0c99-2c2c-482c-ba2a-59ef644afcc9`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:05:21,173::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:05:21,173::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:05:21,173::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:05:21,173::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:05:21,174::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:05:21,174::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:05:21,174::task::993::Storage.TaskManager.Task::(_decref) Task=`092e0c99-2c2c-482c-ba2a-59ef644afcc9`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:05:21,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:21,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115772 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:23,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:23,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181849 s, 56.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:25,381::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) (190451) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) (190451) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190451) Building response >Thread-14::DEBUG::2014-08-22 09:05:25,382::__init__::225::IOProcess::(_processLogs) (190451) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:25,383::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:25,383::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:05:25,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:05:25,411::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000213617 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:25,412::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:25,413::__init__::225::IOProcess::(_processLogs) (190452) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:25,413::__init__::225::IOProcess::(_processLogs) (190452) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:25,413::__init__::225::IOProcess::(_processLogs) (190452) Building response >Thread-14::DEBUG::2014-08-22 09:05:25,413::__init__::225::IOProcess::(_processLogs) (190452) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:25,413::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:05:25,413::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:05:25,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:25,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153194 s, 66.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:05:25,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`31151dcf-b37c-451a-ae35-bccc1e33591e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:25,729::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:05:25,730::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000213617', 'lastCheck': '0.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000224242', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:25,730::task::1191::Storage.TaskManager.Task::(prepare) Task=`31151dcf-b37c-451a-ae35-bccc1e33591e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000213617', 'lastCheck': '0.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000224242', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:25,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`31151dcf-b37c-451a-ae35-bccc1e33591e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:25,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:25,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:25,730::task::993::Storage.TaskManager.Task::(_decref) Task=`31151dcf-b37c-451a-ae35-bccc1e33591e`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:05:25,757::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:25,990::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:25,991::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:25,991::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:27,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:27,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117842 s, 86.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:28,834::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:28,835::__init__::225::IOProcess::(_processLogs) (190453) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:28,835::__init__::225::IOProcess::(_processLogs) (190453) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:28,835::__init__::225::IOProcess::(_processLogs) (190453) Building response >Thread-14::DEBUG::2014-08-22 09:05:28,835::__init__::225::IOProcess::(_processLogs) (190453) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:28,835::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:28,835::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:05:28,836::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:05:28,864::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000264592 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:28,864::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) (190454) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) (190454) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) (190454) Building response >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) (190454) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:28,865::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:28,866::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:05:28,866::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:28,866::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) (190455) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) (190455) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) (190455) Building response >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) (190455) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:28,867::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:28,868::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) (190456) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) (190456) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) (190456) Building response >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) (190456) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:28,869::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) (190457) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) (190457) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) (190457) Building response >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) (190457) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:28,870::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:05:29,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:29,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120358 s, 85.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:30,997::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:30,997::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:30,998::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:31,250::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:05:31,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`47a77597-422d-4af0-91da-b0225e911027`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:31,250::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:05:31,251::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:31,251::task::1191::Storage.TaskManager.Task::(prepare) Task=`47a77597-422d-4af0-91da-b0225e911027`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:31,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`47a77597-422d-4af0-91da-b0225e911027`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:31,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:31,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:31,251::task::993::Storage.TaskManager.Task::(_decref) Task=`47a77597-422d-4af0-91da-b0225e911027`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:05:31,289::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:05:31,289::task::595::Storage.TaskManager.Task::(_updateState) Task=`efb7e603-1b75-4e3f-9c13-6dc051a34777`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:05:31,289::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:05:31,289::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c281507b-d74a-4899-b018-cf79283b6a22`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:05:31,290::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:05:31,290::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:05:31,290::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c281507b-d74a-4899-b018-cf79283b6a22`::Granted request >Thread-171::DEBUG::2014-08-22 09:05:31,290::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`efb7e603-1b75-4e3f-9c13-6dc051a34777`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:05:31,290::task::993::Storage.TaskManager.Task::(_decref) Task=`efb7e603-1b75-4e3f-9c13-6dc051a34777`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:05:31,292::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:31,292::task::1191::Storage.TaskManager.Task::(prepare) Task=`efb7e603-1b75-4e3f-9c13-6dc051a34777`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:31,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`efb7e603-1b75-4e3f-9c13-6dc051a34777`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:05:31,293::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:05:31,293::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:05:31,293::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:05:31,293::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:05:31,293::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:05:31,293::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:05:31,293::task::993::Storage.TaskManager.Task::(_decref) Task=`efb7e603-1b75-4e3f-9c13-6dc051a34777`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:05:31,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:31,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119706 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:33,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:33,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115102 s, 89.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:35,414::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:35,414::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) (190458) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) (190458) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190458) Building response >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) (190458) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:35,415::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:05:35,416::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:05:35,444::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219521 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:35,445::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:35,446::__init__::225::IOProcess::(_processLogs) (190459) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:35,446::__init__::225::IOProcess::(_processLogs) (190459) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:35,446::__init__::225::IOProcess::(_processLogs) (190459) Building response >Thread-14::DEBUG::2014-08-22 09:05:35,446::__init__::225::IOProcess::(_processLogs) (190459) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:35,446::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:35,446::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:05:35,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:35,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120453 s, 85.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:36,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:36,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:36,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:37,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:38,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118533 s, 86.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) (190460) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:38,872::__init__::225::IOProcess::(_processLogs) (190460) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:38,873::__init__::225::IOProcess::(_processLogs) (190460) Building response >Thread-14::DEBUG::2014-08-22 09:05:38,873::__init__::225::IOProcess::(_processLogs) (190460) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:38,873::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:38,873::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:05:38,873::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:05:38,902::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000301149 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:38,902::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:38,902::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:38,902::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) (190461) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) (190461) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) (190461) Building response >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) (190461) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:38,903::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:05:38,904::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:38,904::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:05:38,904::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:38,904::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) (190462) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) (190462) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) (190462) Building response >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) (190462) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:38,905::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) (190463) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:38,906::__init__::225::IOProcess::(_processLogs) (190463) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) (190463) Building response >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) (190463) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:05:38,907::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) (190464) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) (190464) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) (190464) Building response >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) (190464) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:38,908::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:05:40,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:40,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120815 s, 84.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:05:40,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`e96b24f5-90d6-46db-b24c-eeab5ebe0f7a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:40,952::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:05:40,952::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219521', 'lastCheck': '5.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000301149', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:40,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`e96b24f5-90d6-46db-b24c-eeab5ebe0f7a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219521', 'lastCheck': '5.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000301149', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:40,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`e96b24f5-90d6-46db-b24c-eeab5ebe0f7a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:40,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:40,952::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:40,952::task::993::Storage.TaskManager.Task::(_decref) Task=`e96b24f5-90d6-46db-b24c-eeab5ebe0f7a`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:05:40,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:41,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:41,011::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:41,011::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:41,376::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:05:41,376::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ce183ee-3d71-4237-bc7f-c8da54a4d7b6`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:41,376::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:05:41,377::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:41,377::task::1191::Storage.TaskManager.Task::(prepare) Task=`8ce183ee-3d71-4237-bc7f-c8da54a4d7b6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:41,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ce183ee-3d71-4237-bc7f-c8da54a4d7b6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:41,377::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:41,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:41,377::task::993::Storage.TaskManager.Task::(_decref) Task=`8ce183ee-3d71-4237-bc7f-c8da54a4d7b6`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:05:41,401::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:05:41,401::task::595::Storage.TaskManager.Task::(_updateState) Task=`17d7f26d-7866-4dcb-9d61-1e1f6ada5e7d`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:05:41,401::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:05:41,401::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ed9bf09f-90fe-4cd1-9f3d-77508034d49b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:05:41,401::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:05:41,401::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:05:41,402::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ed9bf09f-90fe-4cd1-9f3d-77508034d49b`::Granted request >Thread-171::DEBUG::2014-08-22 09:05:41,402::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`17d7f26d-7866-4dcb-9d61-1e1f6ada5e7d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:05:41,402::task::993::Storage.TaskManager.Task::(_decref) Task=`17d7f26d-7866-4dcb-9d61-1e1f6ada5e7d`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:05:41,404::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:41,404::task::1191::Storage.TaskManager.Task::(prepare) Task=`17d7f26d-7866-4dcb-9d61-1e1f6ada5e7d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:41,404::task::595::Storage.TaskManager.Task::(_updateState) Task=`17d7f26d-7866-4dcb-9d61-1e1f6ada5e7d`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:05:41,404::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:05:41,404::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:05:41,404::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:05:41,405::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:05:41,405::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:05:41,405::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:05:41,405::task::993::Storage.TaskManager.Task::(_decref) Task=`17d7f26d-7866-4dcb-9d61-1e1f6ada5e7d`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:05:42,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:42,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115569 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:44,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:44,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116513 s, 87.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:45,447::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:45,447::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:45,447::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) (190465) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) (190465) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) (190465) Building response >Thread-14::DEBUG::2014-08-22 09:05:45,448::__init__::225::IOProcess::(_processLogs) (190465) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:45,449::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:05:45,449::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:05:45,449::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:05:45,478::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196163 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:45,478::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:45,478::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) (190466) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) (190466) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) (190466) Building response >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) (190466) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:45,479::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:46,017::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:46,017::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:46,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:46,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:46,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118656 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:48,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:48,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118032 s, 86.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:48,909::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) (190467) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) (190467) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) (190467) Building response >Thread-14::DEBUG::2014-08-22 09:05:48,910::__init__::225::IOProcess::(_processLogs) (190467) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:48,911::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:48,911::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:05:48,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:05:48,939::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000245154 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:48,940::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:48,940::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:48,940::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:48,940::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) (190468) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) (190468) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) (190468) Building response >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) (190468) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:48,941::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:05:48,942::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:48,942::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:05:48,942::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:48,942::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) (190469) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) (190469) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) (190469) Building response >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) (190469) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:48,943::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) (190470) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:48,944::__init__::225::IOProcess::(_processLogs) (190470) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:48,945::__init__::225::IOProcess::(_processLogs) (190470) Building response >Thread-14::DEBUG::2014-08-22 09:05:48,945::__init__::225::IOProcess::(_processLogs) (190470) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:48,945::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:48,945::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:48,945::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:48,945::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) (190471) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) (190471) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) (190471) Building response >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) (190471) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:48,946::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:05:50,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:50,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117614 s, 87.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:51,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:51,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:51,025::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:51,454::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:05:51,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`2557f373-cdac-4bd8-a21c-5d591bbbe1e4`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:51,454::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:05:51,455::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:51,455::task::1191::Storage.TaskManager.Task::(prepare) Task=`2557f373-cdac-4bd8-a21c-5d591bbbe1e4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:05:51,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`2557f373-cdac-4bd8-a21c-5d591bbbe1e4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:51,455::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:51,455::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:51,455::task::993::Storage.TaskManager.Task::(_decref) Task=`2557f373-cdac-4bd8-a21c-5d591bbbe1e4`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:05:51,480::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:05:51,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ae0b9ad-642d-4e88-8fb1-2e1f54a64b4b`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:05:51,480::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:05:51,481::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3f02d92e-9e9b-4d73-8780-a66cd8224dc4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:05:51,481::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:05:51,481::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:05:51,481::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3f02d92e-9e9b-4d73-8780-a66cd8224dc4`::Granted request >Thread-171::DEBUG::2014-08-22 09:05:51,481::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5ae0b9ad-642d-4e88-8fb1-2e1f54a64b4b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:05:51,481::task::993::Storage.TaskManager.Task::(_decref) Task=`5ae0b9ad-642d-4e88-8fb1-2e1f54a64b4b`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:05:51,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:51,483::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ae0b9ad-642d-4e88-8fb1-2e1f54a64b4b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:05:51,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ae0b9ad-642d-4e88-8fb1-2e1f54a64b4b`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:05:51,483::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:05:51,484::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:05:51,484::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:05:51,484::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:05:51,484::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:05:51,484::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:05:51,484::task::993::Storage.TaskManager.Task::(_decref) Task=`5ae0b9ad-642d-4e88-8fb1-2e1f54a64b4b`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:05:52,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:52,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117084 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:54,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:54,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0128978 s, 79.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) (190472) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:55,481::__init__::225::IOProcess::(_processLogs) (190472) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:55,482::__init__::225::IOProcess::(_processLogs) (190472) Building response >Thread-14::DEBUG::2014-08-22 09:05:55,482::__init__::225::IOProcess::(_processLogs) (190472) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:55,482::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:55,482::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:05:55,482::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:05:55,511::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000275316 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) (190473) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) (190473) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:55,512::__init__::225::IOProcess::(_processLogs) (190473) Building response >Thread-14::DEBUG::2014-08-22 09:05:55,513::__init__::225::IOProcess::(_processLogs) (190473) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:55,513::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:55,513::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:56,030::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:56,031::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:05:56,031::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:05:56,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfb2237d-3fb1-483b-9802-22dfba3f29fd`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:05:56,152::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:05:56,152::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000275316', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245154', 'lastCheck': '7.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:56,152::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfb2237d-3fb1-483b-9802-22dfba3f29fd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000275316', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245154', 'lastCheck': '7.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:05:56,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfb2237d-3fb1-483b-9802-22dfba3f29fd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:05:56,153::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:05:56,153::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:05:56,153::task::993::Storage.TaskManager.Task::(_decref) Task=`bfb2237d-3fb1-483b-9802-22dfba3f29fd`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:05:56,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:05:56,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:56,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118509 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:05:58,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:05:58,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116937 s, 87.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:58,963::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) (190474) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) (190474) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:58,964::__init__::225::IOProcess::(_processLogs) (190474) Building response >Thread-14::DEBUG::2014-08-22 09:05:58,965::__init__::225::IOProcess::(_processLogs) (190474) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:58,965::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:58,965::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:05:58,965::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:05:59,029::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000188022 s, 3.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:05:59,029::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:59,029::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:05:59,029::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) (190475) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) (190475) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) (190475) Building response >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) (190475) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:59,030::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:05:59,031::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:59,031::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:05:59,031::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:59,031::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) (190476) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) (190476) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) (190476) Building response >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) (190476) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:59,032::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) (190477) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:59,033::__init__::225::IOProcess::(_processLogs) (190477) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:59,034::__init__::225::IOProcess::(_processLogs) (190477) Building response >Thread-14::DEBUG::2014-08-22 09:05:59,034::__init__::225::IOProcess::(_processLogs) (190477) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:59,034::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:59,034::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:05:59,034::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) (190478) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) (190478) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) (190478) Building response >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) (190478) Queuing response >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:05:59,035::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:06:00,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:00,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121208 s, 84.5 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:01,037::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:01,038::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:01,038::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:01,566::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:06:01,566::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f8baa72-d4e8-4917-9db7-88295eb5b42a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:01,566::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:06:01,567::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:01,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f8baa72-d4e8-4917-9db7-88295eb5b42a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:01,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f8baa72-d4e8-4917-9db7-88295eb5b42a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:01,567::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:01,567::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:01,567::task::993::Storage.TaskManager.Task::(_decref) Task=`6f8baa72-d4e8-4917-9db7-88295eb5b42a`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:06:01,600::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:06:01,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`d23ec8db-85d0-4980-9372-bd9dbfb4cabf`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:06:01,600::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:06:01,600::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cbdce586-b680-4a9a-a2cd-404eaa8deb8e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:06:01,600::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:06:01,601::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:06:01,601::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cbdce586-b680-4a9a-a2cd-404eaa8deb8e`::Granted request >Thread-171::DEBUG::2014-08-22 09:06:01,601::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d23ec8db-85d0-4980-9372-bd9dbfb4cabf`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:06:01,601::task::993::Storage.TaskManager.Task::(_decref) Task=`d23ec8db-85d0-4980-9372-bd9dbfb4cabf`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:06:01,603::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:01,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`d23ec8db-85d0-4980-9372-bd9dbfb4cabf`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:01,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`d23ec8db-85d0-4980-9372-bd9dbfb4cabf`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:06:01,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:06:01,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:06:01,604::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:06:01,604::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:06:01,604::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:06:01,604::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:06:01,604::task::993::Storage.TaskManager.Task::(_decref) Task=`d23ec8db-85d0-4980-9372-bd9dbfb4cabf`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:02,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:02,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117148 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:04,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:04,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122002 s, 83.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:05,514::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:05,514::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:05,514::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:05,514::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:05,514::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:05,514::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) (190479) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) (190479) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190479) Building response >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) (190479) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:05,515::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:06:05,516::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:06:05,544::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000291133 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:05,544::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:05,544::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) (190480) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) (190480) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) (190480) Building response >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) (190480) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:05,545::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:06,044::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:06,045::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:06,045::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:06,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:07,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136566 s, 75.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) (190481) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) (190481) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) (190481) Building response >Thread-14::DEBUG::2014-08-22 09:06:09,037::__init__::225::IOProcess::(_processLogs) (190481) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:09,038::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:06:09,038::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:06:09,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:09,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-22 09:06:09,067::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000182967 s, 3.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:09,096::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:09,096::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:09,096::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:09,096::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:09,096::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:09,096::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:09,097::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:09,097::__init__::225::IOProcess::(_processLogs) (190482) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:09,097::__init__::225::IOProcess::(_processLogs) (190482) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:09,105::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190482) Building response >Thread-14::DEBUG::2014-08-22 09:06:09,106::__init__::225::IOProcess::(_processLogs) (190482) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:09,107::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:06:09,107::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:06:09,129::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:09,130::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:06:09,131::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:09,132::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:09,132::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:09,133::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:09,134::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:09,135::__init__::225::IOProcess::(_processLogs) (190483) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:09,135::__init__::225::IOProcess::(_processLogs) (190483) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:09,136::__init__::225::IOProcess::(_processLogs) (190483) Building response >Thread-14::DEBUG::2014-08-22 09:06:09,137::__init__::225::IOProcess::(_processLogs) (190483) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:09,137::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:09,138::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:06:09,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0445574 s, 23.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:09,184::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:09,185::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:06:09,186::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:09,186::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:09,187::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:09,188::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:09,188::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:09,189::__init__::225::IOProcess::(_processLogs) (190484) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:09,190::__init__::225::IOProcess::(_processLogs) (190484) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:09,190::__init__::225::IOProcess::(_processLogs) (190484) Building response >Thread-14::DEBUG::2014-08-22 09:06:09,191::__init__::225::IOProcess::(_processLogs) (190484) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:09,191::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:09,192::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:09,195::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:09,195::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:06:09,196::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:09,197::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:09,197::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:09,198::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:09,199::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:09,200::__init__::225::IOProcess::(_processLogs) (190485) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:09,200::__init__::225::IOProcess::(_processLogs) (190485) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:09,201::__init__::225::IOProcess::(_processLogs) (190485) Building response >Thread-14::DEBUG::2014-08-22 09:06:09,202::__init__::225::IOProcess::(_processLogs) (190485) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:09,203::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:09,204::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:11,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:11,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:11,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:11,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:11,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118072 s, 86.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:06:11,361::task::595::Storage.TaskManager.Task::(_updateState) Task=`46ac512b-66f4-423b-9082-e9ac92fd5778`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:11,361::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:06:11,362::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000291133', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182967', 'lastCheck': '2.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:11,362::task::1191::Storage.TaskManager.Task::(prepare) Task=`46ac512b-66f4-423b-9082-e9ac92fd5778`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000291133', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182967', 'lastCheck': '2.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:11,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`46ac512b-66f4-423b-9082-e9ac92fd5778`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:11,362::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:11,362::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:11,362::task::993::Storage.TaskManager.Task::(_decref) Task=`46ac512b-66f4-423b-9082-e9ac92fd5778`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:06:11,387::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:11,662::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:06:11,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d429c77-7d34-4ae3-89c7-b2295724b9f1`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:11,662::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:06:11,662::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:11,662::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d429c77-7d34-4ae3-89c7-b2295724b9f1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:11,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d429c77-7d34-4ae3-89c7-b2295724b9f1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:11,663::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:11,663::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:11,663::task::993::Storage.TaskManager.Task::(_decref) Task=`1d429c77-7d34-4ae3-89c7-b2295724b9f1`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:06:11,687::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:06:11,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`443032c5-f20c-4139-9949-60d7e4d9bd13`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:06:11,687::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:06:11,687::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`272a7ab2-4392-4b97-bc05-a4d39592992e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:06:11,688::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:06:11,688::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:06:11,688::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`272a7ab2-4392-4b97-bc05-a4d39592992e`::Granted request >Thread-171::DEBUG::2014-08-22 09:06:11,688::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`443032c5-f20c-4139-9949-60d7e4d9bd13`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:06:11,688::task::993::Storage.TaskManager.Task::(_decref) Task=`443032c5-f20c-4139-9949-60d7e4d9bd13`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:06:11,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:11,690::task::1191::Storage.TaskManager.Task::(prepare) Task=`443032c5-f20c-4139-9949-60d7e4d9bd13`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:11,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`443032c5-f20c-4139-9949-60d7e4d9bd13`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:06:11,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:06:11,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:06:11,691::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:06:11,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:06:11,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:06:11,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:06:11,691::task::993::Storage.TaskManager.Task::(_decref) Task=`443032c5-f20c-4139-9949-60d7e4d9bd13`::ref 0 aborting False >Thread-183::DEBUG::2014-08-22 09:06:12,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`dfdfb909-0ac9-48da-8854-105abf06e51a`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:06:12,706::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:06:12,706::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:12,706::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:06:12,706::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:12,706::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) (190486) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) (190486) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|(190486) Building response >Thread-14::DEBUG::2014-08-22 09:06:12,707::__init__::225::IOProcess::(_processLogs) (190486) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:12,708::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:06:12,708::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:06:12,708::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:12,708::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:06:12,708::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) (190487) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) (190487) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) (190487) Building response >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) (190487) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:12,709::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:06:12,710::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:06:12,710::task::1191::Storage.TaskManager.Task::(prepare) Task=`dfdfb909-0ac9-48da-8854-105abf06e51a`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:06:12,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`dfdfb909-0ac9-48da-8854-105abf06e51a`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:06:12,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:06:12,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:06:12,710::task::993::Storage.TaskManager.Task::(_decref) Task=`dfdfb909-0ac9-48da-8854-105abf06e51a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:13,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:13,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139778 s, 73.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:15,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:15,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119882 s, 85.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) (190488) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:15,547::__init__::225::IOProcess::(_processLogs) (190488) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:15,548::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|(190488) Building response >Thread-14::DEBUG::2014-08-22 09:06:15,548::__init__::225::IOProcess::(_processLogs) (190488) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:15,548::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:15,548::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:06:15,548::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:06:15,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187737 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:15,578::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:15,578::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:15,578::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) (190489) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) (190489) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) (190489) Building response >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) (190489) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:15,579::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:16,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:16,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:16,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:17,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:17,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120616 s, 84.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:19,206::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:19,207::__init__::225::IOProcess::(_processLogs) (190490) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:19,207::__init__::225::IOProcess::(_processLogs) (190490) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:19,207::__init__::225::IOProcess::(_processLogs) (190490) Building response >Thread-14::DEBUG::2014-08-22 09:06:19,207::__init__::225::IOProcess::(_processLogs) (190490) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:19,207::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:19,207::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:06:19,208::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:06:19,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234086 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:19,236::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) (190491) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) (190491) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) (190491) Building response >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) (190491) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:19,237::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:19,238::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:06:19,238::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:19,238::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) (190492) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) (190492) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) (190492) Building response >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) (190492) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:19,239::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:19,240::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:19,240::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:06:19,240::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:19,240::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:19,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:19,240::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) (190493) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) (190493) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) (190493) Building response >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) (190493) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:19,241::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) (190494) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) (190494) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) (190494) Building response >Thread-14::DEBUG::2014-08-22 09:06:19,242::__init__::225::IOProcess::(_processLogs) (190494) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:19,243::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:19,243::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:06:19,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:19,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021787 s, 47.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:21,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:21,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:21,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:21,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-13::DEBUG::2014-08-22 09:06:21,749::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:06:21,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eb09217-39d8-42f0-b675-6351771dc4fa`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:21,750::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:06:21,751::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:21,751::task::1191::Storage.TaskManager.Task::(prepare) Task=`1eb09217-39d8-42f0-b675-6351771dc4fa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:21,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eb09217-39d8-42f0-b675-6351771dc4fa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:21,751::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:21,751::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:21,751::task::993::Storage.TaskManager.Task::(_decref) Task=`1eb09217-39d8-42f0-b675-6351771dc4fa`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:21,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124361 s, 82.3 MB/s\n'; <rc> = 0 >Thread-171::DEBUG::2014-08-22 09:06:21,783::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:06:21,784::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a82a731-311a-427f-9b3a-f76b90121639`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:06:21,785::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:06:21,786::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`97652782-5896-4ceb-975a-198a5fce423b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:06:21,786::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:06:21,787::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:06:21,788::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`97652782-5896-4ceb-975a-198a5fce423b`::Granted request >Thread-171::DEBUG::2014-08-22 09:06:21,789::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6a82a731-311a-427f-9b3a-f76b90121639`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:06:21,789::task::993::Storage.TaskManager.Task::(_decref) Task=`6a82a731-311a-427f-9b3a-f76b90121639`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:06:21,798::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:21,798::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a82a731-311a-427f-9b3a-f76b90121639`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:21,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a82a731-311a-427f-9b3a-f76b90121639`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:06:21,799::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:06:21,799::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:06:21,800::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:06:21,801::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:06:21,801::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:06:21,801::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:06:21,802::task::993::Storage.TaskManager.Task::(_decref) Task=`6a82a731-311a-427f-9b3a-f76b90121639`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:23,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:23,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118473 s, 86.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:25,581::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:25,582::__init__::225::IOProcess::(_processLogs) (190495) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:25,582::__init__::225::IOProcess::(_processLogs) (190495) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:25,582::__init__::225::IOProcess::(_processLogs) (190495) Building response >Thread-14::DEBUG::2014-08-22 09:06:25,582::__init__::225::IOProcess::(_processLogs) (190495) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:25,582::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:06:25,582::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:06:25,583::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:06:25,611::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000327817 s, 1.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:25,611::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:25,611::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) (190496) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) (190496) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) (190496) Building response >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) (190496) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:25,612::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:06:25,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:25,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121916 s, 84.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:26,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:26,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:26,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:26,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`be4a0022-6c6b-43ef-ade5-ea21906c05ee`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:26,598::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:06:26,599::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000327817', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234086', 'lastCheck': '7.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:26,599::task::1191::Storage.TaskManager.Task::(prepare) Task=`be4a0022-6c6b-43ef-ade5-ea21906c05ee`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000327817', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234086', 'lastCheck': '7.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:26,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`be4a0022-6c6b-43ef-ade5-ea21906c05ee`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:26,599::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:26,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:26,599::task::993::Storage.TaskManager.Task::(_decref) Task=`be4a0022-6c6b-43ef-ade5-ea21906c05ee`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:06:26,623::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:27,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:28,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118045 s, 86.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) (190497) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:29,244::__init__::225::IOProcess::(_processLogs) (190497) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:29,245::__init__::225::IOProcess::(_processLogs) DEBUG|(190497) Building response >Thread-14::DEBUG::2014-08-22 09:06:29,245::__init__::225::IOProcess::(_processLogs) (190497) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:29,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:29,245::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:06:29,245::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:06:29,273::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000240904 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:29,274::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:29,274::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:29,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) (190498) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) (190498) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) (190498) Building response >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) (190498) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:29,275::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:06:29,276::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:29,276::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:06:29,276::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) (190499) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) (190499) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) (190499) Building response >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) (190499) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:29,277::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) (190500) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:29,278::__init__::225::IOProcess::(_processLogs) (190500) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) (190500) Building response >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) (190500) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:06:29,279::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) (190501) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) (190501) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) (190501) Building response >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) (190501) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:29,280::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:06:30,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:30,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119418 s, 85.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:31,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:31,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:31,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:31,851::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:06:31,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc176d87-5806-4113-9957-ed6a476a120d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:31,852::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:06:31,852::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:31,852::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc176d87-5806-4113-9957-ed6a476a120d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:31,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc176d87-5806-4113-9957-ed6a476a120d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:31,852::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:31,852::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:31,852::task::993::Storage.TaskManager.Task::(_decref) Task=`bc176d87-5806-4113-9957-ed6a476a120d`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:06:31,887::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:06:31,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3407a6d-6b9b-4732-83ac-f25e7ae28be6`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:06:31,887::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:06:31,888::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`064ec33d-e497-4f38-823a-725dcc0c4dc3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:06:31,888::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:06:31,888::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:06:31,888::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`064ec33d-e497-4f38-823a-725dcc0c4dc3`::Granted request >Thread-171::DEBUG::2014-08-22 09:06:31,888::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b3407a6d-6b9b-4732-83ac-f25e7ae28be6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:06:31,888::task::993::Storage.TaskManager.Task::(_decref) Task=`b3407a6d-6b9b-4732-83ac-f25e7ae28be6`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:06:31,890::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:31,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`b3407a6d-6b9b-4732-83ac-f25e7ae28be6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:31,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3407a6d-6b9b-4732-83ac-f25e7ae28be6`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:06:31,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:06:31,891::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:06:31,891::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:06:31,891::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:06:31,891::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:06:31,891::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:06:31,891::task::993::Storage.TaskManager.Task::(_decref) Task=`b3407a6d-6b9b-4732-83ac-f25e7ae28be6`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:32,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:32,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115904 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:34,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:34,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117432 s, 87.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:35,614::__init__::225::IOProcess::(_processLogs) (190502) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:35,615::__init__::225::IOProcess::(_processLogs) (190502) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:35,615::__init__::225::IOProcess::(_processLogs) DEBUG|(190502) Building response >Thread-14::DEBUG::2014-08-22 09:06:35,615::__init__::225::IOProcess::(_processLogs) (190502) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:35,615::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:35,615::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:06:35,615::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:06:35,643::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00024918 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:35,644::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:35,644::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:35,644::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:35,644::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:35,644::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:35,644::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) (190503) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) (190503) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) (190503) Building response >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) (190503) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:35,645::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:36,085::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:36,085::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:36,085::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:36,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:36,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118136 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:38,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:38,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116601 s, 87.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:39,281::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:39,281::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) (190504) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) (190504) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) (190504) Building response >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) (190504) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:39,282::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:06:39,283::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:06:39,311::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000237961 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:39,312::__init__::225::IOProcess::(_processLogs) (190505) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:39,313::__init__::225::IOProcess::(_processLogs) (190505) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:39,313::__init__::225::IOProcess::(_processLogs) (190505) Building response >Thread-14::DEBUG::2014-08-22 09:06:39,313::__init__::225::IOProcess::(_processLogs) (190505) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:39,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:39,313::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:39,314::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) (190506) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) (190506) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) (190506) Building response >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) (190506) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:39,315::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) (190507) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) (190507) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) (190507) Building response >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) (190507) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:39,316::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:39,317::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) (190508) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) (190508) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) (190508) Building response >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) (190508) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:39,318::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:06:40,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:40,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118924 s, 86.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:41,091::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:41,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:41,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:41,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`05cd6bca-0426-49be-80bc-5c27663b1cc3`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:41,824::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:06:41,824::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00024918', 'lastCheck': '6.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237961', 'lastCheck': '2.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:41,824::task::1191::Storage.TaskManager.Task::(prepare) Task=`05cd6bca-0426-49be-80bc-5c27663b1cc3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00024918', 'lastCheck': '6.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237961', 'lastCheck': '2.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:41,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`05cd6bca-0426-49be-80bc-5c27663b1cc3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:41,824::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:41,824::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:41,824::task::993::Storage.TaskManager.Task::(_decref) Task=`05cd6bca-0426-49be-80bc-5c27663b1cc3`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:06:41,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:41,948::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:06:41,948::task::595::Storage.TaskManager.Task::(_updateState) Task=`860d4bae-acf8-421f-811e-33df18c00f97`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:41,948::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:06:41,949::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:41,949::task::1191::Storage.TaskManager.Task::(prepare) Task=`860d4bae-acf8-421f-811e-33df18c00f97`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:41,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`860d4bae-acf8-421f-811e-33df18c00f97`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:41,949::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:41,949::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:41,949::task::993::Storage.TaskManager.Task::(_decref) Task=`860d4bae-acf8-421f-811e-33df18c00f97`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:06:41,973::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:06:41,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7fd77c8-ed6c-44ad-8545-c2a0d3113376`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:06:41,973::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:06:41,974::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6625ca0c-22bf-4a8f-9c2f-bbf42adcb9d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:06:41,974::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:06:41,974::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:06:41,974::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6625ca0c-22bf-4a8f-9c2f-bbf42adcb9d4`::Granted request >Thread-171::DEBUG::2014-08-22 09:06:41,974::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e7fd77c8-ed6c-44ad-8545-c2a0d3113376`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:06:41,974::task::993::Storage.TaskManager.Task::(_decref) Task=`e7fd77c8-ed6c-44ad-8545-c2a0d3113376`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:06:41,976::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:41,976::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7fd77c8-ed6c-44ad-8545-c2a0d3113376`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:41,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7fd77c8-ed6c-44ad-8545-c2a0d3113376`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:06:41,977::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:06:41,977::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:06:41,977::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:06:41,977::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:06:41,977::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:06:41,977::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:06:41,977::task::993::Storage.TaskManager.Task::(_decref) Task=`e7fd77c8-ed6c-44ad-8545-c2a0d3113376`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:42,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:42,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118546 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:44,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:44,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115184 s, 88.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:45,646::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:45,646::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) (190509) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) (190509) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) (190509) Building response >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) (190509) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:45,647::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:06:45,648::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:06:45,676::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000218152 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:45,677::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:45,678::__init__::225::IOProcess::(_processLogs) (190510) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:45,678::__init__::225::IOProcess::(_processLogs) (190510) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:45,678::__init__::225::IOProcess::(_processLogs) (190510) Building response >Thread-14::DEBUG::2014-08-22 09:06:45,678::__init__::225::IOProcess::(_processLogs) (190510) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:45,678::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:45,678::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:46,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:46,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:46,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:46,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:46,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121671 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:48,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:48,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011663 s, 87.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:49,319::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:49,319::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:49,319::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:49,319::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) (190511) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) (190511) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) (190511) Building response >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) (190511) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:49,320::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:06:49,321::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:06:49,349::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000232613 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) (190512) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:49,350::__init__::225::IOProcess::(_processLogs) (190512) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:49,351::__init__::225::IOProcess::(_processLogs) (190512) Building response >Thread-14::DEBUG::2014-08-22 09:06:49,351::__init__::225::IOProcess::(_processLogs) (190512) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:49,351::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:49,351::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) (190513) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:49,352::__init__::225::IOProcess::(_processLogs) (190513) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) (190513) Building response >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) (190513) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:06:49,353::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) (190514) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) (190514) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) (190514) Building response >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) (190514) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:49,354::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:49,355::__init__::225::IOProcess::(_processLogs) (190515) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:49,356::__init__::225::IOProcess::(_processLogs) (190515) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:49,356::__init__::225::IOProcess::(_processLogs) (190515) Building response >Thread-14::DEBUG::2014-08-22 09:06:49,356::__init__::225::IOProcess::(_processLogs) (190515) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:49,356::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:49,356::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:06:50,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:50,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121622 s, 84.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:51,105::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:51,105::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:51,106::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:06:52,027::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:06:52,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`15309848-4700-43c9-b2a5-d1fa5eef73a9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:52,028::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:06:52,028::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:52,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`15309848-4700-43c9-b2a5-d1fa5eef73a9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:06:52,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`15309848-4700-43c9-b2a5-d1fa5eef73a9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:52,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:52,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:52,029::task::993::Storage.TaskManager.Task::(_decref) Task=`15309848-4700-43c9-b2a5-d1fa5eef73a9`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:06:52,108::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:06:52,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`e52ea156-24b0-489a-9f98-1ad78da4c53a`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:06:52,109::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:06:52,109::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`88e6714e-4db0-4567-8c8d-931aedb25242`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:06:52,109::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:06:52,109::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:06:52,109::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`88e6714e-4db0-4567-8c8d-931aedb25242`::Granted request >Thread-171::DEBUG::2014-08-22 09:06:52,110::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e52ea156-24b0-489a-9f98-1ad78da4c53a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:06:52,110::task::993::Storage.TaskManager.Task::(_decref) Task=`e52ea156-24b0-489a-9f98-1ad78da4c53a`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:06:52,112::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:52,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`e52ea156-24b0-489a-9f98-1ad78da4c53a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:06:52,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`e52ea156-24b0-489a-9f98-1ad78da4c53a`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:06:52,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:06:52,112::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:06:52,112::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:06:52,112::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:06:52,112::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:06:52,113::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:06:52,113::task::993::Storage.TaskManager.Task::(_decref) Task=`e52ea156-24b0-489a-9f98-1ad78da4c53a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:06:52,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:52,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116514 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:06:54,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:54,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116558 s, 87.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:55,679::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:55,679::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) (190516) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) (190516) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190516) Building response >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) (190516) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:55,680::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:06:55,681::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:06:55,709::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000303928 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:55,709::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:55,709::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:06:55,709::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) (190517) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) (190517) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) (190517) Building response >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) (190517) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:55,710::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:56,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:56,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:06:56,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:56,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:57,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118857 s, 86.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:06:57,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`3326bbca-263f-44de-a7ca-28e173c8d7bd`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:06:57,057::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:06:57,057::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000303928', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000232613', 'lastCheck': '7.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:57,058::task::1191::Storage.TaskManager.Task::(prepare) Task=`3326bbca-263f-44de-a7ca-28e173c8d7bd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000303928', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000232613', 'lastCheck': '7.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:06:57,058::task::595::Storage.TaskManager.Task::(_updateState) Task=`3326bbca-263f-44de-a7ca-28e173c8d7bd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:06:57,058::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:06:57,059::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:06:57,059::task::993::Storage.TaskManager.Task::(_decref) Task=`3326bbca-263f-44de-a7ca-28e173c8d7bd`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:06:57,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:06:59,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:06:59,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115356 s, 88.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:59,357::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:59,357::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:59,357::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:59,357::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:59,357::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:59,357::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) (190518) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) (190518) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190518) Building response >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) (190518) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:59,358::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:06:59,359::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:06:59,387::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00023778 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) (190519) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) (190519) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:06:59,388::__init__::225::IOProcess::(_processLogs) (190519) Building response >Thread-14::DEBUG::2014-08-22 09:06:59,389::__init__::225::IOProcess::(_processLogs) (190519) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:59,389::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:59,389::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:06:59,389::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) (190520) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) (190520) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) (190520) Building response >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) (190520) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:59,390::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:59,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) (190521) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) (190521) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) (190521) Building response >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) (190521) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:06:59,392::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) (190522) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) (190522) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) (190522) Building response >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) (190522) Queuing response >Thread-14::DEBUG::2014-08-22 09:06:59,393::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:06:59,394::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:01,118::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:01,119::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:01,119::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:01,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:01,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118632 s, 86.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:02,154::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:07:02,155::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b8e4610-5575-4a22-89db-5d10de0dcd1e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:02,155::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:07:02,155::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:02,155::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b8e4610-5575-4a22-89db-5d10de0dcd1e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:02,155::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b8e4610-5575-4a22-89db-5d10de0dcd1e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:02,155::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:02,156::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:02,156::task::993::Storage.TaskManager.Task::(_decref) Task=`3b8e4610-5575-4a22-89db-5d10de0dcd1e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:07:02,209::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:07:02,210::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0689cc2-58e3-42e5-8699-40928f93ad9a`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:07:02,210::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:07:02,210::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9cec1a22-43f2-4ee1-a5e4-a165226c1769`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:07:02,210::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:07:02,210::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:07:02,210::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9cec1a22-43f2-4ee1-a5e4-a165226c1769`::Granted request >Thread-171::DEBUG::2014-08-22 09:07:02,211::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a0689cc2-58e3-42e5-8699-40928f93ad9a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:07:02,211::task::993::Storage.TaskManager.Task::(_decref) Task=`a0689cc2-58e3-42e5-8699-40928f93ad9a`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:07:02,213::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:02,213::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0689cc2-58e3-42e5-8699-40928f93ad9a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:02,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0689cc2-58e3-42e5-8699-40928f93ad9a`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:07:02,213::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:07:02,213::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:07:02,213::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:07:02,213::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:07:02,213::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:07:02,214::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:07:02,214::task::993::Storage.TaskManager.Task::(_decref) Task=`a0689cc2-58e3-42e5-8699-40928f93ad9a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:07:03,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:03,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116846 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:07:05,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:05,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115939 s, 88.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:05,712::__init__::225::IOProcess::(_processLogs) (190523) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:05,713::__init__::225::IOProcess::(_processLogs) (190523) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:05,713::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190523) Building response >Thread-14::DEBUG::2014-08-22 09:07:05,713::__init__::225::IOProcess::(_processLogs) (190523) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:05,713::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:05,713::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:07:05,713::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:07:05,741::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000232823 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:05,742::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:05,742::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:05,742::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:05,742::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) (190524) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) (190524) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) (190524) Building response >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) (190524) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:05,743::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:06,125::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:06,125::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:06,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:07,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:07,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118092 s, 86.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) (190525) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:09,395::__init__::225::IOProcess::(_processLogs) (190525) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:09,396::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190525) Building response >Thread-14::DEBUG::2014-08-22 09:07:09,396::__init__::225::IOProcess::(_processLogs) (190525) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:09,396::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:09,396::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:07:09,396::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:07:09,425::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000312892 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:09,425::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:09,425::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) (190526) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) (190526) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) (190526) Building response >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) (190526) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:09,426::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:07:09,427::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:09,427::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) (190527) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) (190527) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) (190527) Building response >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) (190527) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:09,428::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:09,429::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) (190528) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) (190528) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) (190528) Building response >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) (190528) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:09,430::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) (190529) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) (190529) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) (190529) Building response >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) (190529) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:09,431::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:07:09,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:09,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117945 s, 86.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:11,132::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:11,132::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:11,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:11,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:11,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118552 s, 86.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:12,284::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:07:12,284::task::595::Storage.TaskManager.Task::(_updateState) Task=`43e26b95-faa5-4b88-9cc9-e637fb695fd9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:12,284::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:07:12,285::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:12,285::task::1191::Storage.TaskManager.Task::(prepare) Task=`43e26b95-faa5-4b88-9cc9-e637fb695fd9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:12,285::task::595::Storage.TaskManager.Task::(_updateState) Task=`43e26b95-faa5-4b88-9cc9-e637fb695fd9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:12,285::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:12,285::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:12,285::task::993::Storage.TaskManager.Task::(_decref) Task=`43e26b95-faa5-4b88-9cc9-e637fb695fd9`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:07:12,322::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:07:12,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f12a9f4-289d-4073-b934-699d7a9a3349`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:07:12,322::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:07:12,322::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`835d56a7-b647-4715-8702-93277171a2e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:07:12,322::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:07:12,322::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:07:12,323::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`835d56a7-b647-4715-8702-93277171a2e9`::Granted request >Thread-171::DEBUG::2014-08-22 09:07:12,324::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2f12a9f4-289d-4073-b934-699d7a9a3349`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:07:12,324::task::993::Storage.TaskManager.Task::(_decref) Task=`2f12a9f4-289d-4073-b934-699d7a9a3349`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:07:12,326::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:12,326::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f12a9f4-289d-4073-b934-699d7a9a3349`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:12,326::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f12a9f4-289d-4073-b934-699d7a9a3349`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:07:12,326::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:07:12,326::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:07:12,327::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:07:12,327::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:07:12,327::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:07:12,327::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:07:12,327::task::993::Storage.TaskManager.Task::(_decref) Task=`2f12a9f4-289d-4073-b934-699d7a9a3349`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:07:12,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`3581010c-4153-4913-873a-b42cb853fffa`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:12,395::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:07:12,395::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232823', 'lastCheck': '6.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000312892', 'lastCheck': '3.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:12,395::task::1191::Storage.TaskManager.Task::(prepare) Task=`3581010c-4153-4913-873a-b42cb853fffa`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232823', 'lastCheck': '6.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000312892', 'lastCheck': '3.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:12,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`3581010c-4153-4913-873a-b42cb853fffa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:12,395::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:12,395::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:12,395::task::993::Storage.TaskManager.Task::(_decref) Task=`3581010c-4153-4913-873a-b42cb853fffa`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:07:12,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-183::DEBUG::2014-08-22 09:07:12,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b459f52-8a8a-453c-80f3-4e385d74d6ad`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:07:12,759::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:07:12,759::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:12,759::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:07:12,759::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) (190530) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) (190530) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:07:12,760::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190530) Building response >Thread-14::DEBUG::2014-08-22 09:07:12,761::__init__::225::IOProcess::(_processLogs) (190530) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:12,761::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:07:12,761::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:07:12,761::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:12,761::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) (190531) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) (190531) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) (190531) Building response >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) (190531) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:12,762::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:07:12,763::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:07:12,763::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b459f52-8a8a-453c-80f3-4e385d74d6ad`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:07:12,763::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b459f52-8a8a-453c-80f3-4e385d74d6ad`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:07:12,763::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:07:12,763::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:07:12,763::task::993::Storage.TaskManager.Task::(_decref) Task=`4b459f52-8a8a-453c-80f3-4e385d74d6ad`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:07:13,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:13,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119015 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:07:15,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:15,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132842 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) (190532) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) (190532) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:15,745::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190532) Building response >Thread-14::DEBUG::2014-08-22 09:07:15,746::__init__::225::IOProcess::(_processLogs) (190532) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:15,746::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:15,746::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:07:15,746::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:07:15,775::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000313854 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:15,775::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:15,775::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) (190533) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) (190533) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) (190533) Building response >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) (190533) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:15,776::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:16,139::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:16,139::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:16,140::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:17,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:17,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118987 s, 86.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:19,433::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:19,434::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:19,434::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:19,435::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:19,435::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:19,436::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:19,436::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:19,436::__init__::225::IOProcess::(_processLogs) (190534) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:19,437::__init__::225::IOProcess::(_processLogs) (190534) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:19,437::__init__::225::IOProcess::(_processLogs) (190534) Building response >Thread-14::DEBUG::2014-08-22 09:07:19,438::__init__::225::IOProcess::(_processLogs) (190534) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:19,438::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:19,439::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:07:19,441::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:07:19,470::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200078 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:19,471::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:19,472::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:19,473::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:19,473::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:19,473::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:19,474::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:19,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:19,475::__init__::225::IOProcess::(_processLogs) (190535) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:19,475::__init__::225::IOProcess::(_processLogs) (190535) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:19,476::__init__::225::IOProcess::(_processLogs) (190535) Building response >Thread-14::DEBUG::2014-08-22 09:07:19,476::__init__::225::IOProcess::(_processLogs) (190535) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:19,477::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:19,477::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:07:19,480::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:19,481::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:07:19,481::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:19,482::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:19,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:19,483::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:19,483::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:19,484::__init__::225::IOProcess::(_processLogs) (190536) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:19,484::__init__::225::IOProcess::(_processLogs) (190536) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:19,485::__init__::225::IOProcess::(_processLogs) (190536) Building response >Thread-14::DEBUG::2014-08-22 09:07:19,485::__init__::225::IOProcess::(_processLogs) (190536) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:19,485::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:19,486::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:19,488::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:19,488::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:07:19,488::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:19,489::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:19,489::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:19,490::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:19,490::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:19,491::__init__::225::IOProcess::(_processLogs) (190537) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:19,491::__init__::225::IOProcess::(_processLogs) (190537) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:19,492::__init__::225::IOProcess::(_processLogs) (190537) Building response >Thread-14::DEBUG::2014-08-22 09:07:19,492::__init__::225::IOProcess::(_processLogs) (190537) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:19,493::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:19,493::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:19,495::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:19,495::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:07:19,496::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:19,496::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:19,496::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:19,497::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:19,497::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:19,498::__init__::225::IOProcess::(_processLogs) (190538) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:19,498::__init__::225::IOProcess::(_processLogs) (190538) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:19,499::__init__::225::IOProcess::(_processLogs) (190538) Building response >Thread-14::DEBUG::2014-08-22 09:07:19,499::__init__::225::IOProcess::(_processLogs) (190538) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:19,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:19,500::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:07:19,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:19,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180014 s, 56.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:21,145::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:21,146::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:21,146::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:21,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:22,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119521 s, 85.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:22,384::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:07:22,384::task::595::Storage.TaskManager.Task::(_updateState) Task=`260afd65-12c9-4e52-8d82-1af50275e44e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:22,384::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:07:22,384::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:22,384::task::1191::Storage.TaskManager.Task::(prepare) Task=`260afd65-12c9-4e52-8d82-1af50275e44e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:22,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`260afd65-12c9-4e52-8d82-1af50275e44e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:22,385::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:22,385::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:22,385::task::993::Storage.TaskManager.Task::(_decref) Task=`260afd65-12c9-4e52-8d82-1af50275e44e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:07:22,434::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:07:22,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef682c5f-c887-4fb9-82f3-52e69ba7601c`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:07:22,434::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:07:22,435::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`77d13a74-04fe-40ec-86f2-4042f92cfaa1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:07:22,435::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:07:22,435::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:07:22,435::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`77d13a74-04fe-40ec-86f2-4042f92cfaa1`::Granted request >Thread-171::DEBUG::2014-08-22 09:07:22,435::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ef682c5f-c887-4fb9-82f3-52e69ba7601c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:07:22,435::task::993::Storage.TaskManager.Task::(_decref) Task=`ef682c5f-c887-4fb9-82f3-52e69ba7601c`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:07:22,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:22,438::task::1191::Storage.TaskManager.Task::(prepare) Task=`ef682c5f-c887-4fb9-82f3-52e69ba7601c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:22,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef682c5f-c887-4fb9-82f3-52e69ba7601c`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:07:22,438::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:07:22,438::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:07:22,438::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:07:22,438::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:07:22,438::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:07:22,438::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:07:22,438::task::993::Storage.TaskManager.Task::(_decref) Task=`ef682c5f-c887-4fb9-82f3-52e69ba7601c`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:07:24,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:24,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011522 s, 88.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:25,778::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:25,779::__init__::225::IOProcess::(_processLogs) (190539) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:25,779::__init__::225::IOProcess::(_processLogs) (190539) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:25,779::__init__::225::IOProcess::(_processLogs) (190539) Building response >Thread-14::DEBUG::2014-08-22 09:07:25,779::__init__::225::IOProcess::(_processLogs) (190539) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:25,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:25,779::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:07:25,780::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:07:25,808::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000331369 s, 1.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) (190540) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) (190540) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:25,809::__init__::225::IOProcess::(_processLogs) (190540) Building response >Thread-14::DEBUG::2014-08-22 09:07:25,810::__init__::225::IOProcess::(_processLogs) (190540) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:25,810::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:25,810::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:07:26,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:26,163::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:26,163::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:26,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:26,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119259 s, 85.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:27,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5a45cce-b43d-448b-989b-8059854828c9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:27,705::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:07:27,705::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000331369', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000200078', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:27,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5a45cce-b43d-448b-989b-8059854828c9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000331369', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000200078', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:27,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5a45cce-b43d-448b-989b-8059854828c9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:27,706::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:27,706::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:27,706::task::993::Storage.TaskManager.Task::(_decref) Task=`d5a45cce-b43d-448b-989b-8059854828c9`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:07:27,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:28,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:28,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011611 s, 88.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-22 09:07:29,502::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-14::DEBUG::2014-08-22 09:07:29,503::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,503::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-14::DEBUG::2014-08-22 09:07:29,503::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) (190541) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) (190541) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) (190541) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) (190541) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,504::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-14::DEBUG::2014-08-22 09:07:29,505::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,505::__init__::225::IOProcess::(_processLogs) Message size is 145 >Thread-14::DEBUG::2014-08-22 09:07:29,505::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,505::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,505::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) (190542) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) (190542) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190542) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) (190542) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,506::__init__::225::IOProcess::(_processLogs) Sending response sized 195 >Thread-16::DEBUG::2014-08-22 09:07:29,508::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-14::DEBUG::2014-08-22 09:07:29,508::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) Message size is 174 >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) (190543) Finding callback 'touch'... >Thread-14::DEBUG::2014-08-22 09:07:29,509::__init__::225::IOProcess::(_processLogs) (190543) Got request for method 'touch' >Thread-14::DEBUG::2014-08-22 09:07:29,532::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190543) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,533::__init__::225::IOProcess::(_processLogs) (190543) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,533::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,533::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-16::DEBUG::2014-08-22 09:07:29,533::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-22 09:07:29,533::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,533::__init__::225::IOProcess::(_processLogs) Message size is 198 >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) (190544) Finding callback 'readfile'... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) (190544) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190544) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) (190544) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,534::__init__::225::IOProcess::(_processLogs) Sending response sized 834 >Thread-16::DEBUG::2014-08-22 09:07:29,535::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-14::DEBUG::2014-08-22 09:07:29,535::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,535::__init__::225::IOProcess::(_processLogs) Message size is 191 >Thread-14::DEBUG::2014-08-22 09:07:29,535::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) (190545) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) (190545) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) (190545) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) (190545) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,536::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) (190546) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:07:29,537::__init__::225::IOProcess::(_processLogs) (190546) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:07:29,538::__init__::225::IOProcess::(_processLogs) (190546) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,538::__init__::225::IOProcess::(_processLogs) (190546) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,538::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-16::DEBUG::2014-08-22 09:07:29,538::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-22 09:07:29,538::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-16::INFO::2014-08-22 09:07:29,538::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) (190547) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) (190547) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) (190547) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,539::__init__::225::IOProcess::(_processLogs) (190547) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,540::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,540::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:07:29,540::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:07:29,569::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00040265 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:29,569::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,569::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:29,569::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) (190548) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) (190548) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) (190548) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) (190548) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,570::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:07:29,571::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,571::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:07:29,571::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,571::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) (190549) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) (190549) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) (190549) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) (190549) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,572::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) (190550) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) (190550) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:29,573::__init__::225::IOProcess::(_processLogs) (190550) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) (190550) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:29,574::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) (190551) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) (190551) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) (190551) Building response >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) (190551) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:29,575::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:07:30,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:30,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118494 s, 86.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:31,169::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:31,170::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:31,170::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:32,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:32,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116289 s, 88.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:32,537::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:07:32,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`13a9a0f0-29dc-459c-a879-587156abafdc`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:32,538::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:07:32,538::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:32,538::task::1191::Storage.TaskManager.Task::(prepare) Task=`13a9a0f0-29dc-459c-a879-587156abafdc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:32,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`13a9a0f0-29dc-459c-a879-587156abafdc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:32,538::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:32,538::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:32,539::task::993::Storage.TaskManager.Task::(_decref) Task=`13a9a0f0-29dc-459c-a879-587156abafdc`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:07:32,571::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:07:32,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb718377-3727-44ca-88d3-61e0c8749f61`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:07:32,571::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:07:32,572::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`88498a31-ca2c-4f30-8354-5e617e5f4d65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:07:32,572::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:07:32,572::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:07:32,572::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`88498a31-ca2c-4f30-8354-5e617e5f4d65`::Granted request >Thread-171::DEBUG::2014-08-22 09:07:32,572::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bb718377-3727-44ca-88d3-61e0c8749f61`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:07:32,572::task::993::Storage.TaskManager.Task::(_decref) Task=`bb718377-3727-44ca-88d3-61e0c8749f61`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:07:32,575::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:32,575::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb718377-3727-44ca-88d3-61e0c8749f61`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:32,575::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb718377-3727-44ca-88d3-61e0c8749f61`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:07:32,575::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:07:32,575::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:07:32,575::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:07:32,575::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:07:32,575::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:07:32,575::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:07:32,575::task::993::Storage.TaskManager.Task::(_decref) Task=`bb718377-3727-44ca-88d3-61e0c8749f61`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:07:34,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:34,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118814 s, 86.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:35,811::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:35,811::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:35,811::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:35,811::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) (190552) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) (190552) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190552) Building response >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) (190552) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:35,812::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:07:35,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:07:35,841::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000226973 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) (190553) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) (190553) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:35,842::__init__::225::IOProcess::(_processLogs) (190553) Building response >Thread-14::DEBUG::2014-08-22 09:07:35,843::__init__::225::IOProcess::(_processLogs) (190553) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:35,843::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:35,843::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:36,176::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:36,177::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:36,177::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:36,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:36,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119345 s, 85.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:07:38,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:38,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117917 s, 86.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:39,576::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:39,576::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:39,576::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) (190554) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) (190554) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190554) Building response >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) (190554) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:39,577::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:07:39,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:07:39,607::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000176391 s, 3.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:39,608::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:39,609::__init__::225::IOProcess::(_processLogs) (190555) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:39,609::__init__::225::IOProcess::(_processLogs) (190555) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:39,609::__init__::225::IOProcess::(_processLogs) (190555) Building response >Thread-14::DEBUG::2014-08-22 09:07:39,609::__init__::225::IOProcess::(_processLogs) (190555) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:39,609::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:39,609::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:39,610::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) (190556) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) (190556) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) (190556) Building response >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) (190556) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:39,611::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) (190557) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) (190557) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) (190557) Building response >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) (190557) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:39,612::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:39,613::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) (190558) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) (190558) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) (190558) Building response >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) (190558) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:39,614::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:07:40,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:40,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119756 s, 85.5 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:41,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:41,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:41,184::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:07:42,667::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:07:42,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba980cf6-c928-4454-983c-6d8ad886b588`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:42,668::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:07:42,668::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:42,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba980cf6-c928-4454-983c-6d8ad886b588`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:42,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba980cf6-c928-4454-983c-6d8ad886b588`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:42,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:42,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:42,668::task::993::Storage.TaskManager.Task::(_decref) Task=`ba980cf6-c928-4454-983c-6d8ad886b588`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:07:42,709::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:07:42,709::task::595::Storage.TaskManager.Task::(_updateState) Task=`f072158f-1ad6-4547-91d9-d04b7721897a`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:07:42,709::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:07:42,710::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`55abad1b-2c5d-443b-9e01-d8db70c42764`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:07:42,710::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:07:42,710::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:07:42,710::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`55abad1b-2c5d-443b-9e01-d8db70c42764`::Granted request >Thread-171::DEBUG::2014-08-22 09:07:42,710::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f072158f-1ad6-4547-91d9-d04b7721897a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:07:42,710::task::993::Storage.TaskManager.Task::(_decref) Task=`f072158f-1ad6-4547-91d9-d04b7721897a`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:07:42,712::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:42,713::task::1191::Storage.TaskManager.Task::(prepare) Task=`f072158f-1ad6-4547-91d9-d04b7721897a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:42,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`f072158f-1ad6-4547-91d9-d04b7721897a`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:07:42,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:07:42,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:07:42,713::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:07:42,713::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:07:42,713::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:07:42,713::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:07:42,713::task::993::Storage.TaskManager.Task::(_decref) Task=`f072158f-1ad6-4547-91d9-d04b7721897a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:07:42,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:42,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117443 s, 87.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:43,041::task::595::Storage.TaskManager.Task::(_updateState) Task=`e913bb12-62fc-4d99-ba2b-9fa760aeeed2`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:43,042::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:07:43,042::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000226973', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000176391', 'lastCheck': '3.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:43,042::task::1191::Storage.TaskManager.Task::(prepare) Task=`e913bb12-62fc-4d99-ba2b-9fa760aeeed2`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000226973', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000176391', 'lastCheck': '3.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:43,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`e913bb12-62fc-4d99-ba2b-9fa760aeeed2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:43,042::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:43,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:43,042::task::993::Storage.TaskManager.Task::(_decref) Task=`e913bb12-62fc-4d99-ba2b-9fa760aeeed2`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:07:43,067::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:44,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:44,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01156 s, 88.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:45,844::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:45,844::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:45,844::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:45,844::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) (190559) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) (190559) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) (190559) Building response >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) (190559) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:45,845::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:07:45,846::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:07:45,874::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000226444 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:45,874::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) (190560) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) (190560) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) (190560) Building response >Thread-14::DEBUG::2014-08-22 09:07:45,875::__init__::225::IOProcess::(_processLogs) (190560) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:45,876::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:45,876::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:46,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:46,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:46,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:46,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:46,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121236 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:07:48,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:49,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116413 s, 88.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:49,616::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:49,617::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:49,617::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:49,618::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:49,619::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:49,619::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:49,620::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:49,621::__init__::225::IOProcess::(_processLogs) (190561) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:49,621::__init__::225::IOProcess::(_processLogs) (190561) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:49,622::__init__::225::IOProcess::(_processLogs) (190561) Building response >Thread-14::DEBUG::2014-08-22 09:07:49,622::__init__::225::IOProcess::(_processLogs) (190561) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:49,623::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:49,624::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:07:49,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:07:49,656::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000263144 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:49,658::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:49,659::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:49,660::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:49,660::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:49,661::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:49,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:49,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:49,663::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190562) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:49,664::__init__::225::IOProcess::(_processLogs) (190562) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:49,665::__init__::225::IOProcess::(_processLogs) (190562) Building response >Thread-14::DEBUG::2014-08-22 09:07:49,665::__init__::225::IOProcess::(_processLogs) (190562) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:49,666::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:49,666::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:07:49,671::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:49,671::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:07:49,672::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:49,673::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:49,674::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:49,674::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:49,675::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:49,676::__init__::225::IOProcess::(_processLogs) (190563) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:49,676::__init__::225::IOProcess::(_processLogs) (190563) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:49,677::__init__::225::IOProcess::(_processLogs) (190563) Building response >Thread-14::DEBUG::2014-08-22 09:07:49,678::__init__::225::IOProcess::(_processLogs) (190563) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:49,678::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:49,679::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:49,681::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:49,682::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:07:49,683::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:49,684::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:49,684::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:49,685::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:49,686::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:49,687::__init__::225::IOProcess::(_processLogs) (190564) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:49,687::__init__::225::IOProcess::(_processLogs) (190564) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:49,688::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190564) Building response >Thread-14::DEBUG::2014-08-22 09:07:49,689::__init__::225::IOProcess::(_processLogs) (190564) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:49,690::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:49,691::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:49,693::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:49,694::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:07:49,695::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:49,696::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:49,696::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:49,697::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:49,698::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:49,699::__init__::225::IOProcess::(_processLogs) (190565) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:49,700::__init__::225::IOProcess::(_processLogs) (190565) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:49,701::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190565) Building response >Thread-14::DEBUG::2014-08-22 09:07:49,702::__init__::225::IOProcess::(_processLogs) (190565) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:49,702::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:49,703::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:07:51,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:51,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122269 s, 83.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:51,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:51,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:51,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:07:52,787::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:07:52,787::task::595::Storage.TaskManager.Task::(_updateState) Task=`128e6be1-9b4b-449c-b871-228d9d334582`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:52,787::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:07:52,787::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:52,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`128e6be1-9b4b-449c-b871-228d9d334582`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:07:52,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`128e6be1-9b4b-449c-b871-228d9d334582`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:52,788::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:52,788::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:52,788::task::993::Storage.TaskManager.Task::(_decref) Task=`128e6be1-9b4b-449c-b871-228d9d334582`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:07:52,821::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:07:52,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6152d9c-6401-409e-b376-60629455e837`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:07:52,821::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:07:52,821::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`dbe66282-66b6-45bb-80e9-0688d335e5c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:07:52,822::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:07:52,822::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:07:52,822::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`dbe66282-66b6-45bb-80e9-0688d335e5c4`::Granted request >Thread-171::DEBUG::2014-08-22 09:07:52,822::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a6152d9c-6401-409e-b376-60629455e837`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:07:52,822::task::993::Storage.TaskManager.Task::(_decref) Task=`a6152d9c-6401-409e-b376-60629455e837`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:07:52,824::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:52,824::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6152d9c-6401-409e-b376-60629455e837`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:07:52,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6152d9c-6401-409e-b376-60629455e837`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:07:52,824::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:07:52,825::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:07:52,825::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:07:52,825::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:07:52,825::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:07:52,825::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:07:52,825::task::993::Storage.TaskManager.Task::(_decref) Task=`a6152d9c-6401-409e-b376-60629455e837`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:07:53,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:53,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116754 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:07:55,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:55,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115032 s, 89.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:55,877::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:55,877::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:55,877::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:55,877::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:55,877::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) (190566) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) (190566) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) (190566) Building response >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) (190566) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:55,878::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:07:55,879::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:07:55,907::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000197645 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) (190567) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) (190567) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:55,908::__init__::225::IOProcess::(_processLogs) (190567) Building response >Thread-14::DEBUG::2014-08-22 09:07:55,909::__init__::225::IOProcess::(_processLogs) (190567) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:55,909::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:55,909::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:56,203::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:56,204::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:07:56,204::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:57,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:57,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119302 s, 85.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:07:58,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b5f0fb2-f438-45fa-8570-6476e128c92e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:07:58,298::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:07:58,298::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000197645', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000263144', 'lastCheck': '8.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:58,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b5f0fb2-f438-45fa-8570-6476e128c92e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000197645', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000263144', 'lastCheck': '8.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:07:58,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b5f0fb2-f438-45fa-8570-6476e128c92e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:07:58,299::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:07:58,299::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:07:58,299::task::993::Storage.TaskManager.Task::(_decref) Task=`9b5f0fb2-f438-45fa-8570-6476e128c92e`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:07:58,324::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:07:59,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:07:59,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116855 s, 87.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:59,706::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:59,706::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:59,706::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) (190568) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) (190568) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) (190568) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:59,707::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:07:59,708::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:07:59,736::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234614 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) (190569) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) (190569) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) (190569) Building response >Thread-14::DEBUG::2014-08-22 09:07:59,737::__init__::225::IOProcess::(_processLogs) (190569) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:59,738::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:59,738::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:07:59,738::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) (190570) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) (190570) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) (190570) Building response >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) (190570) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:59,739::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) (190571) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) (190571) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:59,740::__init__::225::IOProcess::(_processLogs) (190571) Building response >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) (190571) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:07:59,741::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) (190572) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) (190572) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) (190572) Building response >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) (190572) Queuing response >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:07:59,742::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:01,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:01,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:01,211::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:01,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:01,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117921 s, 86.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:02,875::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:08:02,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`16c577c0-ae56-4652-8f56-9d5231f085a1`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:02,876::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:08:02,876::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:02,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`16c577c0-ae56-4652-8f56-9d5231f085a1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:02,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`16c577c0-ae56-4652-8f56-9d5231f085a1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:02,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:02,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:02,876::task::993::Storage.TaskManager.Task::(_decref) Task=`16c577c0-ae56-4652-8f56-9d5231f085a1`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:08:02,908::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:08:02,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`613f1279-fe0b-492e-b99c-429643837bd1`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:08:02,908::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:08:02,909::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`653580cf-2b6b-4d84-b533-536d5cb3b34a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:08:02,909::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:08:02,909::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:08:02,909::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`653580cf-2b6b-4d84-b533-536d5cb3b34a`::Granted request >Thread-171::DEBUG::2014-08-22 09:08:02,909::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`613f1279-fe0b-492e-b99c-429643837bd1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:08:02,909::task::993::Storage.TaskManager.Task::(_decref) Task=`613f1279-fe0b-492e-b99c-429643837bd1`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:08:02,911::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:02,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`613f1279-fe0b-492e-b99c-429643837bd1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:02,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`613f1279-fe0b-492e-b99c-429643837bd1`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:08:02,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:08:02,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:08:02,912::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:08:02,912::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:08:02,912::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:08:02,912::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:08:02,912::task::993::Storage.TaskManager.Task::(_decref) Task=`613f1279-fe0b-492e-b99c-429643837bd1`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:08:03,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:03,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115501 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:08:05,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:05,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117269 s, 87.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:05,921::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) (190573) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) (190573) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:05,922::__init__::225::IOProcess::(_processLogs) (190573) Building response >Thread-14::DEBUG::2014-08-22 09:08:05,923::__init__::225::IOProcess::(_processLogs) (190573) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:05,923::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:05,923::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:08:05,923::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:08:05,950::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000143069 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:05,951::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:05,951::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:05,951::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:05,951::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) (190574) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) (190574) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) (190574) Building response >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) (190574) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:05,952::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:06,217::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:06,217::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:06,217::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:07,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:07,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117599 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:08:09,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-22 09:08:09,743::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:09,743::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:09,744::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:09,744::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:09,744::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:09,744::__init__::225::IOProcess::(_processLogs) (190575) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:09,744::__init__::225::IOProcess::(_processLogs) (190575) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:09,745::__init__::225::IOProcess::(_processLogs) (190575) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:09,746::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:08:09,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:08:09,778::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000187421 s, 3.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:09,781::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:09,781::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:09,782::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:09,783::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:09,783::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:09,784::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:09,785::__init__::225::IOProcess::(_processLogs) (190576) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:09,785::__init__::225::IOProcess::(_processLogs) (190576) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:09,786::__init__::225::IOProcess::(_processLogs) (190576) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:09,786::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:09,787::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:08:09,790::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:08:09,791::__init__::225::IOProcess::(_processLogs) Marshaling message... >Dummy-170::DEBUG::2014-08-22 09:08:09,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0544946 s, 18.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:09,816::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:09,816::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:09,817::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:09,817::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:09,818::__init__::225::IOProcess::(_processLogs) (190577) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:09,818::__init__::225::IOProcess::(_processLogs) (190577) Building response >Thread-14::DEBUG::2014-08-22 09:08:09,818::__init__::225::IOProcess::(_processLogs) (190577) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:09,819::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:09,819::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:09,821::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:08:09,821::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:09,821::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:09,822::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:09,822::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:09,823::__init__::225::IOProcess::(_processLogs) (190578) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:09,823::__init__::225::IOProcess::(_processLogs) (190578) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:09,823::__init__::225::IOProcess::(_processLogs) (190578) Building response >Thread-14::DEBUG::2014-08-22 09:08:09,824::__init__::225::IOProcess::(_processLogs) (190578) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:09,824::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:09,825::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:09,826::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:08:09,827::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:09,827::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:09,827::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:09,828::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:09,828::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:09,829::__init__::225::IOProcess::(_processLogs) (190579) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:09,829::__init__::225::IOProcess::(_processLogs) (190579) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:09,829::__init__::225::IOProcess::(_processLogs) (190579) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:09,830::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:09,830::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:11,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:11,224::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:11,224::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:11,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:11,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118759 s, 86.2 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:08:12,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0f21c94-0058-484b-b06e-d7c1f0eb4a11`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:08:12,823::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:08:12,823::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:08:12,823::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) (190580) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) (190580) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) (190580) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:12,824::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:08:12,825::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:08:12,825::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:12,825::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:12,825::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:12,825::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:12,825::__init__::225::IOProcess::(_processLogs) (190581) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:08:12,826::__init__::225::IOProcess::(_processLogs) (190581) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:08:12,826::__init__::225::IOProcess::(_processLogs) (190581) Building response >Thread-14::DEBUG::2014-08-22 09:08:12,826::__init__::225::IOProcess::(_processLogs) (190581) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:12,826::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:12,826::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:08:12,826::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:08:12,826::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0f21c94-0058-484b-b06e-d7c1f0eb4a11`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:08:12,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0f21c94-0058-484b-b06e-d7c1f0eb4a11`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:08:12,827::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:08:12,827::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:08:12,827::task::993::Storage.TaskManager.Task::(_decref) Task=`d0f21c94-0058-484b-b06e-d7c1f0eb4a11`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:12,995::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:08:12,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`438e8853-2f08-418e-8c9a-8ba59e8fb667`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:12,996::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:08:12,996::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:12,996::task::1191::Storage.TaskManager.Task::(prepare) Task=`438e8853-2f08-418e-8c9a-8ba59e8fb667`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:12,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`438e8853-2f08-418e-8c9a-8ba59e8fb667`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:12,996::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:12,996::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:12,996::task::993::Storage.TaskManager.Task::(_decref) Task=`438e8853-2f08-418e-8c9a-8ba59e8fb667`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:08:13,020::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:08:13,020::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e2e3e97-ccf9-4524-bb34-09cecfe256ae`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:08:13,020::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:08:13,020::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`94b5b0e9-452b-445d-8b35-4539c6559951`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:08:13,020::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:08:13,021::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:08:13,021::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`94b5b0e9-452b-445d-8b35-4539c6559951`::Granted request >Thread-171::DEBUG::2014-08-22 09:08:13,021::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3e2e3e97-ccf9-4524-bb34-09cecfe256ae`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:08:13,021::task::993::Storage.TaskManager.Task::(_decref) Task=`3e2e3e97-ccf9-4524-bb34-09cecfe256ae`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:08:13,023::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:13,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e2e3e97-ccf9-4524-bb34-09cecfe256ae`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:13,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e2e3e97-ccf9-4524-bb34-09cecfe256ae`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:08:13,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:08:13,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:08:13,023::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:08:13,024::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:08:13,024::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:08:13,024::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:08:13,024::task::993::Storage.TaskManager.Task::(_decref) Task=`3e2e3e97-ccf9-4524-bb34-09cecfe256ae`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:13,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`210afe37-207d-4e76-88de-2f4b4822a944`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:13,531::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:08:13,532::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000143069', 'lastCheck': '7.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000187421', 'lastCheck': '3.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:13,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`210afe37-207d-4e76-88de-2f4b4822a944`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000143069', 'lastCheck': '7.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000187421', 'lastCheck': '3.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:13,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`210afe37-207d-4e76-88de-2f4b4822a944`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:13,532::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:13,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:13,532::task::993::Storage.TaskManager.Task::(_decref) Task=`210afe37-207d-4e76-88de-2f4b4822a944`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:13,558::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:13,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:13,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195434 s, 52.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:15,953::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:15,953::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) (190582) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) (190582) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) (190582) Building response >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) (190582) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:15,954::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:08:15,955::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:08:15,983::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000207551 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) (190583) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:15,984::__init__::225::IOProcess::(_processLogs) (190583) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:15,985::__init__::225::IOProcess::(_processLogs) (190583) Building response >Thread-14::DEBUG::2014-08-22 09:08:15,985::__init__::225::IOProcess::(_processLogs) (190583) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:15,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:15,985::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:08:15,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:16,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248463 s, 41.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:16,230::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:16,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:16,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:18,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:18,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120956 s, 84.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:19,833::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:19,833::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:19,834::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:19,834::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:19,835::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:19,835::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:19,836::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:19,836::__init__::225::IOProcess::(_processLogs) (190584) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:19,836::__init__::225::IOProcess::(_processLogs) (190584) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:19,837::__init__::225::IOProcess::(_processLogs) (190584) Building response >Thread-14::DEBUG::2014-08-22 09:08:19,837::__init__::225::IOProcess::(_processLogs) (190584) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:19,838::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:19,838::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:08:19,840::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:08:19,869::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000204814 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:19,871::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:19,871::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:19,872::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:19,872::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:19,873::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:19,873::__init__::225::IOProcess::(_processLogs) (190585) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:19,873::__init__::225::IOProcess::(_processLogs) (190585) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:19,874::__init__::225::IOProcess::(_processLogs) (190585) Building response >Thread-14::DEBUG::2014-08-22 09:08:19,874::__init__::225::IOProcess::(_processLogs) (190585) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:19,875::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:19,875::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:08:19,878::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:08:19,879::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:19,879::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:19,879::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:19,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:19,880::__init__::225::IOProcess::(_processLogs) (190586) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:19,881::__init__::225::IOProcess::(_processLogs) (190586) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:19,881::__init__::225::IOProcess::(_processLogs) (190586) Building response >Thread-14::DEBUG::2014-08-22 09:08:19,881::__init__::225::IOProcess::(_processLogs) (190586) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:19,882::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:19,882::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:19,884::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:08:19,884::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:19,885::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:19,885::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:19,886::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:19,886::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:19,887::__init__::225::IOProcess::(_processLogs) (190587) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:19,887::__init__::225::IOProcess::(_processLogs) (190587) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:19,888::__init__::225::IOProcess::(_processLogs) (190587) Building response >Thread-14::DEBUG::2014-08-22 09:08:19,888::__init__::225::IOProcess::(_processLogs) (190587) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:19,888::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:19,889::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:19,891::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:19,891::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:08:19,892::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:19,892::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:19,892::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:19,893::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:19,893::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:19,894::__init__::225::IOProcess::(_processLogs) (190588) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:19,894::__init__::225::IOProcess::(_processLogs) (190588) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:19,895::__init__::225::IOProcess::(_processLogs) (190588) Building response >Thread-14::DEBUG::2014-08-22 09:08:19,895::__init__::225::IOProcess::(_processLogs) (190588) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:19,895::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:19,896::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:08:20,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:20,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017994 s, 56.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:21,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:21,238::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:21,238::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:22,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:22,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118822 s, 86.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:23,073::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:08:23,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`081f2584-394c-4ede-acf5-cf595fdf2fa4`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:23,073::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:08:23,074::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:23,074::task::1191::Storage.TaskManager.Task::(prepare) Task=`081f2584-394c-4ede-acf5-cf595fdf2fa4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:23,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`081f2584-394c-4ede-acf5-cf595fdf2fa4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:23,074::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:23,074::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:23,074::task::993::Storage.TaskManager.Task::(_decref) Task=`081f2584-394c-4ede-acf5-cf595fdf2fa4`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:08:23,109::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:08:23,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c9fc26c-9eb6-47f7-aaf7-9d4e4a88d1cf`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:08:23,110::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:08:23,110::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`70be96ac-d3b7-4235-b5d6-d217a592913b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:08:23,110::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:08:23,110::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:08:23,110::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`70be96ac-d3b7-4235-b5d6-d217a592913b`::Granted request >Thread-171::DEBUG::2014-08-22 09:08:23,112::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5c9fc26c-9eb6-47f7-aaf7-9d4e4a88d1cf`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:08:23,112::task::993::Storage.TaskManager.Task::(_decref) Task=`5c9fc26c-9eb6-47f7-aaf7-9d4e4a88d1cf`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:08:23,114::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:23,114::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c9fc26c-9eb6-47f7-aaf7-9d4e4a88d1cf`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:23,114::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c9fc26c-9eb6-47f7-aaf7-9d4e4a88d1cf`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:08:23,114::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:08:23,114::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:08:23,114::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:08:23,114::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:08:23,114::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:08:23,115::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:08:23,115::task::993::Storage.TaskManager.Task::(_decref) Task=`5c9fc26c-9eb6-47f7-aaf7-9d4e4a88d1cf`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:08:24,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:24,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116559 s, 87.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:25,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:25,986::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:25,986::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:25,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:25,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) (190589) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) (190589) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) (190589) Building response >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) (190589) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:25,987::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:08:25,988::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:08:26,016::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000206608 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) (190590) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) (190590) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:26,017::__init__::225::IOProcess::(_processLogs) (190590) Building response >Thread-14::DEBUG::2014-08-22 09:08:26,018::__init__::225::IOProcess::(_processLogs) (190590) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:26,018::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:26,018::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:26,244::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:26,245::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:26,245::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:26,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:26,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117926 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:08:28,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:28,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118406 s, 86.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:28,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`63922b73-6959-45f6-9e51-2d570fcd74cd`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:28,733::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:08:28,733::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206608', 'lastCheck': '2.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000204814', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:28,733::task::1191::Storage.TaskManager.Task::(prepare) Task=`63922b73-6959-45f6-9e51-2d570fcd74cd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206608', 'lastCheck': '2.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000204814', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:28,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`63922b73-6959-45f6-9e51-2d570fcd74cd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:28,733::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:28,733::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:28,733::task::993::Storage.TaskManager.Task::(_decref) Task=`63922b73-6959-45f6-9e51-2d570fcd74cd`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:28,758::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:08:29,898::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:29,898::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:29,898::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:29,898::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:29,898::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) (190591) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) (190591) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) (190591) Building response >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) (190591) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:29,899::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:08:29,900::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:08:29,928::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000290918 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:29,929::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:29,929::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:29,929::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:29,929::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:29,929::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:29,929::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) (190592) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) (190592) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) (190592) Building response >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) (190592) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:29,930::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:29,931::__init__::225::IOProcess::(_processLogs) (190593) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) (190593) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) (190593) Building response >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) (190593) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:29,932::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) (190594) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) (190594) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) (190594) Building response >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) (190594) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:29,933::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:29,934::__init__::225::IOProcess::(_processLogs) (190595) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:29,935::__init__::225::IOProcess::(_processLogs) (190595) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:29,935::__init__::225::IOProcess::(_processLogs) (190595) Building response >Thread-14::DEBUG::2014-08-22 09:08:29,935::__init__::225::IOProcess::(_processLogs) (190595) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:29,935::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:29,935::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:08:30,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:30,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119142 s, 85.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:31,251::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:31,252::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:31,252::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:32,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:32,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011534 s, 88.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:33,161::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:08:33,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`af3e9629-61f5-44cc-9906-7c9589a46739`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:33,162::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:08:33,162::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:33,162::task::1191::Storage.TaskManager.Task::(prepare) Task=`af3e9629-61f5-44cc-9906-7c9589a46739`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:33,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`af3e9629-61f5-44cc-9906-7c9589a46739`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:33,162::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:33,162::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:33,163::task::993::Storage.TaskManager.Task::(_decref) Task=`af3e9629-61f5-44cc-9906-7c9589a46739`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:08:33,196::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:08:33,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8fd4478-2871-45b9-ac02-e53129fc88a1`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:08:33,196::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:08:33,196::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8fe89975-8d24-47f2-86bb-817314ad5e23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:08:33,196::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:08:33,196::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:08:33,197::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8fe89975-8d24-47f2-86bb-817314ad5e23`::Granted request >Thread-171::DEBUG::2014-08-22 09:08:33,197::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8fd4478-2871-45b9-ac02-e53129fc88a1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:08:33,197::task::993::Storage.TaskManager.Task::(_decref) Task=`a8fd4478-2871-45b9-ac02-e53129fc88a1`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:08:33,199::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:33,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8fd4478-2871-45b9-ac02-e53129fc88a1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:33,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8fd4478-2871-45b9-ac02-e53129fc88a1`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:08:33,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:08:33,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:08:33,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:08:33,199::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:08:33,200::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:08:33,200::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:08:33,200::task::993::Storage.TaskManager.Task::(_decref) Task=`a8fd4478-2871-45b9-ac02-e53129fc88a1`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:08:34,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:34,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013511 s, 75.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:36,019::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:36,019::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:36,019::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:36,019::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:36,019::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) (190596) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) (190596) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190596) Building response >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) (190596) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:36,020::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:08:36,021::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:08:36,048::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000210405 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:36,049::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:36,049::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:36,049::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:36,049::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) (190597) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) (190597) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) (190597) Building response >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) (190597) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:36,050::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:36,258::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:36,259::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:36,259::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:36,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:36,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119096 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:08:38,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:38,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116778 s, 87.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:39,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:39,936::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:39,936::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:39,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:39,936::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:39,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) (190598) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) (190598) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) (190598) Building response >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) (190598) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:39,937::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:08:39,938::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:08:39,966::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000255611 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:39,966::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:39,966::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) (190599) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) (190599) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) (190599) Building response >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) (190599) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:39,967::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:39,968::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:08:39,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:39,968::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) (190600) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) (190600) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) (190600) Building response >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) (190600) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:39,969::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:39,970::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) (190601) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) (190601) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) (190601) Building response >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) (190601) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:39,971::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) (190602) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) (190602) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) (190602) Building response >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) (190602) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:39,972::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:08:40,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:41,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121794 s, 84.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:41,265::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:41,265::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:41,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:43,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:43,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117826 s, 86.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:43,256::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:08:43,257::task::595::Storage.TaskManager.Task::(_updateState) Task=`29739c7f-663c-4e9c-8d9d-23a361038b9e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:43,257::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:08:43,257::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:43,257::task::1191::Storage.TaskManager.Task::(prepare) Task=`29739c7f-663c-4e9c-8d9d-23a361038b9e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:43,257::task::595::Storage.TaskManager.Task::(_updateState) Task=`29739c7f-663c-4e9c-8d9d-23a361038b9e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:43,257::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:43,257::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:43,257::task::993::Storage.TaskManager.Task::(_decref) Task=`29739c7f-663c-4e9c-8d9d-23a361038b9e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:08:43,296::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:08:43,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`832265b2-3df5-4769-b9c1-f0d5a5c8346d`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:08:43,296::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:08:43,297::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9fa48c69-1705-40a4-8e9b-19a476a7e2ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:08:43,297::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:08:43,297::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:08:43,297::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9fa48c69-1705-40a4-8e9b-19a476a7e2ca`::Granted request >Thread-171::DEBUG::2014-08-22 09:08:43,297::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`832265b2-3df5-4769-b9c1-f0d5a5c8346d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:08:43,297::task::993::Storage.TaskManager.Task::(_decref) Task=`832265b2-3df5-4769-b9c1-f0d5a5c8346d`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:08:43,299::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:43,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`832265b2-3df5-4769-b9c1-f0d5a5c8346d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:43,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`832265b2-3df5-4769-b9c1-f0d5a5c8346d`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:08:43,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:08:43,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:08:43,300::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:08:43,300::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:08:43,300::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:08:43,300::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:08:43,300::task::993::Storage.TaskManager.Task::(_decref) Task=`832265b2-3df5-4769-b9c1-f0d5a5c8346d`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:44,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`50aa21d4-0e83-4bdf-8c95-cab892617020`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:44,147::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:08:44,147::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210405', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000255611', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:44,147::task::1191::Storage.TaskManager.Task::(prepare) Task=`50aa21d4-0e83-4bdf-8c95-cab892617020`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210405', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000255611', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:44,148::task::595::Storage.TaskManager.Task::(_updateState) Task=`50aa21d4-0e83-4bdf-8c95-cab892617020`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:44,148::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:44,148::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:44,148::task::993::Storage.TaskManager.Task::(_decref) Task=`50aa21d4-0e83-4bdf-8c95-cab892617020`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:44,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:45,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:45,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117871 s, 86.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:46,051::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) (190603) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:46,052::__init__::225::IOProcess::(_processLogs) (190603) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:46,053::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190603) Building response >Thread-14::DEBUG::2014-08-22 09:08:46,053::__init__::225::IOProcess::(_processLogs) (190603) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:46,053::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:08:46,053::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:08:46,054::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:08:46,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000241773 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:46,082::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:46,082::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) (190604) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) (190604) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) (190604) Building response >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) (190604) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:46,083::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:46,272::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:46,272::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:46,272::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:47,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:47,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119589 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:08:49,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:49,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114734 s, 89.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:49,974::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:49,975::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:49,975::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:49,976::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:49,976::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:49,977::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:49,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:49,978::__init__::225::IOProcess::(_processLogs) (190605) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:49,978::__init__::225::IOProcess::(_processLogs) (190605) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:49,978::__init__::225::IOProcess::(_processLogs) (190605) Building response >Thread-14::DEBUG::2014-08-22 09:08:49,979::__init__::225::IOProcess::(_processLogs) (190605) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:49,979::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:49,980::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:08:49,982::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:08:50,012::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00019242 s, 3.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:50,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:50,014::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:08:50,014::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:50,015::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:50,015::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:50,015::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:50,016::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:50,016::__init__::225::IOProcess::(_processLogs) (190606) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:50,017::__init__::225::IOProcess::(_processLogs) (190606) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:50,017::__init__::225::IOProcess::(_processLogs) (190606) Building response >Thread-14::DEBUG::2014-08-22 09:08:50,018::__init__::225::IOProcess::(_processLogs) (190606) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:50,018::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:50,018::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:08:50,021::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:50,022::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:08:50,022::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:50,023::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:50,023::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:50,024::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:50,024::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:50,025::__init__::225::IOProcess::(_processLogs) (190607) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:50,025::__init__::225::IOProcess::(_processLogs) (190607) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:50,025::__init__::225::IOProcess::(_processLogs) (190607) Building response >Thread-14::DEBUG::2014-08-22 09:08:50,026::__init__::225::IOProcess::(_processLogs) (190607) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:50,026::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:50,027::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:50,028::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:50,029::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:08:50,029::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:50,030::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:50,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:50,031::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:50,031::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:50,032::__init__::225::IOProcess::(_processLogs) (190608) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:50,032::__init__::225::IOProcess::(_processLogs) (190608) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:50,032::__init__::225::IOProcess::(_processLogs) (190608) Building response >Thread-14::DEBUG::2014-08-22 09:08:50,033::__init__::225::IOProcess::(_processLogs) (190608) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:50,033::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:50,034::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:08:50,036::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:50,036::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:08:50,037::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:50,037::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:50,038::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:50,038::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:50,039::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:50,039::__init__::225::IOProcess::(_processLogs) (190609) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:08:50,039::__init__::225::IOProcess::(_processLogs) (190609) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:08:50,040::__init__::225::IOProcess::(_processLogs) (190609) Building response >Thread-14::DEBUG::2014-08-22 09:08:50,040::__init__::225::IOProcess::(_processLogs) (190609) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:50,041::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:50,041::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:51,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:51,279::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:51,279::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:51,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:51,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120981 s, 84.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:53,368::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:08:53,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`2199c203-0c50-4d09-87d7-0153889ec35f`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:53,369::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:08:53,369::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:53,369::task::1191::Storage.TaskManager.Task::(prepare) Task=`2199c203-0c50-4d09-87d7-0153889ec35f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:08:53,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`2199c203-0c50-4d09-87d7-0153889ec35f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:53,369::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:53,370::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:53,370::task::993::Storage.TaskManager.Task::(_decref) Task=`2199c203-0c50-4d09-87d7-0153889ec35f`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:08:53,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-171::DEBUG::2014-08-22 09:08:53,444::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:08:53,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf2118ac-918f-4d38-87a9-c7201bd86907`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:08:53,444::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:08:53,445::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b406e368-686a-4226-a36b-b3cd27a92a07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:08:53,445::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:08:53,445::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:08:53,445::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b406e368-686a-4226-a36b-b3cd27a92a07`::Granted request >Thread-171::DEBUG::2014-08-22 09:08:53,445::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cf2118ac-918f-4d38-87a9-c7201bd86907`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:08:53,446::task::993::Storage.TaskManager.Task::(_decref) Task=`cf2118ac-918f-4d38-87a9-c7201bd86907`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:08:53,448::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:53,448::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf2118ac-918f-4d38-87a9-c7201bd86907`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:08:53,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf2118ac-918f-4d38-87a9-c7201bd86907`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:08:53,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:08:53,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:08:53,448::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:08:53,448::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:08:53,449::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:08:53,449::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:08:53,449::task::993::Storage.TaskManager.Task::(_decref) Task=`cf2118ac-918f-4d38-87a9-c7201bd86907`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:08:53,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115934 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:08:55,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:55,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117938 s, 86.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:56,085::__init__::225::IOProcess::(_processLogs) (190610) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:56,086::__init__::225::IOProcess::(_processLogs) (190610) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:56,086::__init__::225::IOProcess::(_processLogs) (190610) Building response >Thread-14::DEBUG::2014-08-22 09:08:56,086::__init__::225::IOProcess::(_processLogs) (190610) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:56,086::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:56,086::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:08:56,086::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:08:56,115::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.0002338 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:08:56,115::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) (190611) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) (190611) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) (190611) Building response >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) (190611) Queuing response >Thread-14::DEBUG::2014-08-22 09:08:56,116::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:08:56,117::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:56,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:56,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:08:56,286::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:57,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:57,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119334 s, 85.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:08:59,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebf82166-9e5c-4ad3-9f7e-01a500a30767`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:08:59,368::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:08:59,369::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0002338', 'lastCheck': '3.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00019242', 'lastCheck': '9.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:59,369::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebf82166-9e5c-4ad3-9f7e-01a500a30767`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0002338', 'lastCheck': '3.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00019242', 'lastCheck': '9.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:08:59,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebf82166-9e5c-4ad3-9f7e-01a500a30767`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:08:59,369::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:08:59,369::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:08:59,369::task::993::Storage.TaskManager.Task::(_decref) Task=`ebf82166-9e5c-4ad3-9f7e-01a500a30767`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:08:59,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:08:59,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:08:59,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117953 s, 86.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:00,043::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) (190612) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) (190612) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) (190612) Building response >Thread-14::DEBUG::2014-08-22 09:09:00,044::__init__::225::IOProcess::(_processLogs) (190612) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:00,045::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:00,045::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:09:00,045::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:09:00,073::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216951 s, 2.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:00,074::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:00,074::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:00,074::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:00,074::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:00,074::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:00,074::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) (190613) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) (190613) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) (190613) Building response >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) (190613) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:00,075::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:00,076::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) (190614) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) (190614) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) (190614) Building response >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) (190614) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:00,077::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) (190615) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) (190615) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) (190615) Building response >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) (190615) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:00,078::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:00,079::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) (190616) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) (190616) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) (190616) Building response >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) (190616) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:00,080::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:01,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:01,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:01,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:01,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:01,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183233 s, 55.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:03,497::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:09:03,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb2770af-6866-4866-bb34-417f6fce1d5a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:03,498::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:09:03,498::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:03,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb2770af-6866-4866-bb34-417f6fce1d5a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:03,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb2770af-6866-4866-bb34-417f6fce1d5a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:03,498::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:03,498::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:03,498::task::993::Storage.TaskManager.Task::(_decref) Task=`bb2770af-6866-4866-bb34-417f6fce1d5a`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:09:03,532::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:09:03,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c874301-0c03-4686-b71e-51de64d4e4b4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:09:03,532::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:09:03,533::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9a9981ed-d61b-4dc4-ba90-6bb74ba7ce5e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:09:03,533::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:09:03,533::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:09:03,533::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9a9981ed-d61b-4dc4-ba90-6bb74ba7ce5e`::Granted request >Thread-171::DEBUG::2014-08-22 09:09:03,533::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5c874301-0c03-4686-b71e-51de64d4e4b4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:09:03,533::task::993::Storage.TaskManager.Task::(_decref) Task=`5c874301-0c03-4686-b71e-51de64d4e4b4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:09:03,535::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:03,536::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c874301-0c03-4686-b71e-51de64d4e4b4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:03,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c874301-0c03-4686-b71e-51de64d4e4b4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:09:03,536::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:09:03,536::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:09:03,536::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:09:03,536::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:09:03,536::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:09:03,536::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:09:03,536::task::993::Storage.TaskManager.Task::(_decref) Task=`5c874301-0c03-4686-b71e-51de64d4e4b4`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:09:03,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:03,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116727 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:05,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:05,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116826 s, 87.7 MB/s\n'; <rc> = 0 >Thread-91::DEBUG::2014-08-22 09:09:06,117::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-14::DEBUG::2014-08-22 09:09:06,119::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,119::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-14::DEBUG::2014-08-22 09:09:06,119::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) (190617) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) (190617) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190617) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) (190617) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,120::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-14::DEBUG::2014-08-22 09:09:06,121::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,121::__init__::225::IOProcess::(_processLogs) Message size is 145 >Thread-14::DEBUG::2014-08-22 09:09:06,121::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,122::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,123::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,123::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,124::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,125::__init__::225::IOProcess::(_processLogs) (190618) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:09:06,125::__init__::225::IOProcess::(_processLogs) (190618) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:09:06,126::__init__::225::IOProcess::(_processLogs) (190618) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,127::__init__::225::IOProcess::(_processLogs) (190618) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,127::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,128::__init__::225::IOProcess::(_processLogs) Sending response sized 195 >Thread-91::DEBUG::2014-08-22 09:09:06,129::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-14::DEBUG::2014-08-22 09:09:06,130::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,130::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-14::DEBUG::2014-08-22 09:09:06,130::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,131::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,131::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,131::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,131::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,131::__init__::225::IOProcess::(_processLogs) (190619) Finding callback 'touch'... >Thread-14::DEBUG::2014-08-22 09:09:06,131::__init__::225::IOProcess::(_processLogs) (190619) Got request for method 'touch' >Thread-14::DEBUG::2014-08-22 09:09:06,158::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190619) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,158::__init__::225::IOProcess::(_processLogs) (190619) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,158::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,158::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-91::DEBUG::2014-08-22 09:09:06,159::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,159::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,160::__init__::225::IOProcess::(_processLogs) (190620) Finding callback 'readfile'... >Thread-14::DEBUG::2014-08-22 09:09:06,160::__init__::225::IOProcess::(_processLogs) (190620) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-22 09:09:06,160::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190620) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,160::__init__::225::IOProcess::(_processLogs) (190620) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,160::__init__::225::IOProcess::(_processLogs) Sending response sized 606 >Thread-91::DEBUG::2014-08-22 09:09:06,160::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) (190621) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) (190621) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:06,161::__init__::225::IOProcess::(_processLogs) (190621) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,162::__init__::225::IOProcess::(_processLogs) (190621) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,162::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,162::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:06,162::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,162::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-14::DEBUG::2014-08-22 09:09:06,162::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) (190622) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) (190622) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190622) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) (190622) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,163::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-91::DEBUG::2014-08-22 09:09:06,164::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-91::INFO::2014-08-22 09:09:06,164::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-91::INFO::2014-08-22 09:09:06,164::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-14::DEBUG::2014-08-22 09:09:06,164::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,164::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:06,164::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,164::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) (190623) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) (190623) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) (190623) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) (190623) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,165::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:09:06,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:09:06,194::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000291126 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:06,195::__init__::225::IOProcess::(_processLogs) (190624) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:06,196::__init__::225::IOProcess::(_processLogs) (190624) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:06,196::__init__::225::IOProcess::(_processLogs) (190624) Building response >Thread-14::DEBUG::2014-08-22 09:09:06,196::__init__::225::IOProcess::(_processLogs) (190624) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:06,196::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:06,196::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:06,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:06,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:06,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:07,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:08,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120585 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:10,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:10,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115452 s, 88.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:10,095::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:10,096::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:10,097::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:10,097::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:10,098::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:10,098::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:10,099::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:10,100::__init__::225::IOProcess::(_processLogs) (190625) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:10,100::__init__::225::IOProcess::(_processLogs) (190625) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:10,101::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190625) Building response >Thread-14::DEBUG::2014-08-22 09:09:10,102::__init__::225::IOProcess::(_processLogs) (190625) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:10,102::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:10,102::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:09:10,105::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:09:10,133::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206962 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:10,135::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:10,135::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:10,135::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:10,136::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:10,136::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:10,137::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:10,137::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:10,138::__init__::225::IOProcess::(_processLogs) (190626) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:10,138::__init__::225::IOProcess::(_processLogs) (190626) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:10,138::__init__::225::IOProcess::(_processLogs) (190626) Building response >Thread-14::DEBUG::2014-08-22 09:09:10,139::__init__::225::IOProcess::(_processLogs) (190626) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:10,139::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:10,140::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:09:10,142::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:10,143::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:09:10,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:10,144::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:10,144::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:10,144::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:10,145::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:10,145::__init__::225::IOProcess::(_processLogs) (190627) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:10,146::__init__::225::IOProcess::(_processLogs) (190627) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:10,146::__init__::225::IOProcess::(_processLogs) (190627) Building response >Thread-14::DEBUG::2014-08-22 09:09:10,146::__init__::225::IOProcess::(_processLogs) (190627) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:10,147::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:10,147::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:10,148::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:10,149::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:09:10,149::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:10,150::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:10,150::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:10,150::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:10,151::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:10,151::__init__::225::IOProcess::(_processLogs) (190628) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:10,152::__init__::225::IOProcess::(_processLogs) (190628) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:10,152::__init__::225::IOProcess::(_processLogs) (190628) Building response >Thread-14::DEBUG::2014-08-22 09:09:10,152::__init__::225::IOProcess::(_processLogs) (190628) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:10,153::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:10,153::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:10,155::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:10,155::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:09:10,155::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:10,156::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:10,156::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:10,157::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:10,157::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:10,157::__init__::225::IOProcess::(_processLogs) (190629) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:10,158::__init__::225::IOProcess::(_processLogs) (190629) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:10,158::__init__::225::IOProcess::(_processLogs) (190629) Building response >Thread-14::DEBUG::2014-08-22 09:09:10,159::__init__::225::IOProcess::(_processLogs) (190629) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:10,159::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:10,159::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:11,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:11,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:11,306::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:12,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:12,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116429 s, 88.0 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:09:12,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee07b154-ab8d-4169-a1a1-e4d290342143`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:09:12,877::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:09:12,877::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) (190630) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) (190630) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190630) Building response >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) (190630) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:12,878::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:12,879::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:09:12,879::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:12,879::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:09:12,879::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) (190631) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) (190631) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) (190631) Building response >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) (190631) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:12,880::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:09:12,881::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:09:12,881::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee07b154-ab8d-4169-a1a1-e4d290342143`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:09:12,881::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee07b154-ab8d-4169-a1a1-e4d290342143`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:09:12,881::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:09:12,881::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:09:12,881::task::993::Storage.TaskManager.Task::(_decref) Task=`ee07b154-ab8d-4169-a1a1-e4d290342143`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:09:13,584::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:09:13,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`63cac6f8-4de0-4868-8c92-bbba6bb8a7fe`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:13,585::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:09:13,585::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:13,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`63cac6f8-4de0-4868-8c92-bbba6bb8a7fe`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:13,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`63cac6f8-4de0-4868-8c92-bbba6bb8a7fe`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:13,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:13,585::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:13,585::task::993::Storage.TaskManager.Task::(_decref) Task=`63cac6f8-4de0-4868-8c92-bbba6bb8a7fe`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:09:13,636::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:09:13,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ad474b9-f979-4b04-9182-6879dcb386a1`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:09:13,637::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:09:13,637::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23a14110-ebfe-472d-b8ef-4444b74d162f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:09:13,637::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:09:13,637::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:09:13,637::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23a14110-ebfe-472d-b8ef-4444b74d162f`::Granted request >Thread-171::DEBUG::2014-08-22 09:09:13,638::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1ad474b9-f979-4b04-9182-6879dcb386a1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:09:13,638::task::993::Storage.TaskManager.Task::(_decref) Task=`1ad474b9-f979-4b04-9182-6879dcb386a1`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:09:13,640::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:13,640::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ad474b9-f979-4b04-9182-6879dcb386a1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:13,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ad474b9-f979-4b04-9182-6879dcb386a1`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:09:13,640::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:09:13,640::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:09:13,640::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:09:13,640::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:09:13,640::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:09:13,640::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:09:13,641::task::993::Storage.TaskManager.Task::(_decref) Task=`1ad474b9-f979-4b04-9182-6879dcb386a1`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:09:14,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:14,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119563 s, 85.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:14,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcb31a53-dd73-490a-935e-04c442d95efb`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:14,607::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:09:14,607::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000291126', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206962', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:09:14,607::task::1191::Storage.TaskManager.Task::(prepare) Task=`dcb31a53-dd73-490a-935e-04c442d95efb`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000291126', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206962', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:09:14,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcb31a53-dd73-490a-935e-04c442d95efb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:14,607::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:14,607::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:14,607::task::993::Storage.TaskManager.Task::(_decref) Task=`dcb31a53-dd73-490a-935e-04c442d95efb`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:09:14,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:09:16,197::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:16,197::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:16,197::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:16,197::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:16,197::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) (190632) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) (190632) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) (190632) Building response >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) (190632) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:16,198::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:09:16,199::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:09:16,227::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000254063 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:16,228::__init__::225::IOProcess::(_processLogs) (190633) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:16,229::__init__::225::IOProcess::(_processLogs) (190633) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:16,229::__init__::225::IOProcess::(_processLogs) (190633) Building response >Thread-14::DEBUG::2014-08-22 09:09:16,229::__init__::225::IOProcess::(_processLogs) (190633) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:16,229::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:16,229::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:09:16,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:16,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:16,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:16,342::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:16,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119355 s, 85.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:18,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:18,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119194 s, 85.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:20,162::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:20,162::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:20,163::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:20,164::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:20,164::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:20,165::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:20,166::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:20,166::__init__::225::IOProcess::(_processLogs) (190634) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:20,167::__init__::225::IOProcess::(_processLogs) (190634) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:20,167::__init__::225::IOProcess::(_processLogs) (190634) Building response >Thread-14::DEBUG::2014-08-22 09:09:20,168::__init__::225::IOProcess::(_processLogs) (190634) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:20,169::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:20,169::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:09:20,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:09:20,202::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00016173 s, 3.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:20,204::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:20,205::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:20,205::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:20,206::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:20,207::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:20,207::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:20,208::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:20,209::__init__::225::IOProcess::(_processLogs) (190635) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:20,209::__init__::225::IOProcess::(_processLogs) (190635) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:20,210::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190635) Building response >Thread-14::DEBUG::2014-08-22 09:09:20,211::__init__::225::IOProcess::(_processLogs) (190635) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:20,211::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:20,212::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:09:20,216::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:20,217::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:09:20,217::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:20,217::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:20,218::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:20,218::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:20,219::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:20,219::__init__::225::IOProcess::(_processLogs) (190636) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:20,220::__init__::225::IOProcess::(_processLogs) (190636) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:20,220::__init__::225::IOProcess::(_processLogs) (190636) Building response >Thread-14::DEBUG::2014-08-22 09:09:20,221::__init__::225::IOProcess::(_processLogs) (190636) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:20,221::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:20,221::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:20,223::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:20,224::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:09:20,224::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:20,225::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:20,225::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:20,226::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:20,226::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:20,227::__init__::225::IOProcess::(_processLogs) (190637) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:20,227::__init__::225::IOProcess::(_processLogs) (190637) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:20,228::__init__::225::IOProcess::(_processLogs) (190637) Building response >Thread-14::DEBUG::2014-08-22 09:09:20,228::__init__::225::IOProcess::(_processLogs) (190637) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:20,228::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:20,229::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:20,231::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:20,231::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:09:20,231::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:20,232::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:20,232::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:20,233::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:20,233::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:20,234::__init__::225::IOProcess::(_processLogs) (190638) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:20,234::__init__::225::IOProcess::(_processLogs) (190638) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:20,235::__init__::225::IOProcess::(_processLogs) (190638) Building response >Thread-14::DEBUG::2014-08-22 09:09:20,235::__init__::225::IOProcess::(_processLogs) (190638) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:20,236::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:20,236::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:09:20,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:20,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119412 s, 85.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:21,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:21,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:21,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:22,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:22,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117297 s, 87.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:23,689::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:09:23,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0eb2940-dd57-4e6a-88bd-4caf912307fa`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:23,689::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:09:23,690::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:23,690::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0eb2940-dd57-4e6a-88bd-4caf912307fa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:23,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0eb2940-dd57-4e6a-88bd-4caf912307fa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:23,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:23,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:23,690::task::993::Storage.TaskManager.Task::(_decref) Task=`d0eb2940-dd57-4e6a-88bd-4caf912307fa`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:09:23,715::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:09:23,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`25ea6649-b497-4e4f-aa9e-fbea943eb913`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:09:23,715::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:09:23,716::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ad911124-70ac-475a-9cf2-5e30c3608f6d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:09:23,716::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:09:23,716::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:09:23,716::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ad911124-70ac-475a-9cf2-5e30c3608f6d`::Granted request >Thread-171::DEBUG::2014-08-22 09:09:23,716::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`25ea6649-b497-4e4f-aa9e-fbea943eb913`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:09:23,716::task::993::Storage.TaskManager.Task::(_decref) Task=`25ea6649-b497-4e4f-aa9e-fbea943eb913`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:09:23,718::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:23,718::task::1191::Storage.TaskManager.Task::(prepare) Task=`25ea6649-b497-4e4f-aa9e-fbea943eb913`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:23,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`25ea6649-b497-4e4f-aa9e-fbea943eb913`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:09:23,719::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:09:23,719::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:09:23,719::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:09:23,719::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:09:23,719::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:09:23,719::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:09:23,719::task::993::Storage.TaskManager.Task::(_decref) Task=`25ea6649-b497-4e4f-aa9e-fbea943eb913`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:09:24,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:24,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118642 s, 86.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:26,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:26,230::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:26,230::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:26,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:26,230::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) (190639) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) (190639) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) (190639) Building response >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) (190639) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:09:26,231::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:09:26,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:09:26,260::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000223362 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:26,261::__init__::225::IOProcess::(_processLogs) (190640) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:26,262::__init__::225::IOProcess::(_processLogs) (190640) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:26,262::__init__::225::IOProcess::(_processLogs) (190640) Building response >Thread-14::DEBUG::2014-08-22 09:09:26,262::__init__::225::IOProcess::(_processLogs) (190640) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:26,262::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:26,262::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:26,354::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:26,355::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:26,355::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:26,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:26,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119889 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:28,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:28,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114526 s, 89.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:29,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`33c85bd8-839a-4326-8006-e9e523e0605c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:29,817::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:09:29,817::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223362', 'lastCheck': '3.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00016173', 'lastCheck': '9.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:09:29,817::task::1191::Storage.TaskManager.Task::(prepare) Task=`33c85bd8-839a-4326-8006-e9e523e0605c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223362', 'lastCheck': '3.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00016173', 'lastCheck': '9.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:09:29,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`33c85bd8-839a-4326-8006-e9e523e0605c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:29,818::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:29,818::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:29,818::task::993::Storage.TaskManager.Task::(_decref) Task=`33c85bd8-839a-4326-8006-e9e523e0605c`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:09:29,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:09:30,238::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:30,238::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:30,238::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:30,238::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) (190641) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) (190641) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190641) Building response >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) (190641) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:30,239::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:09:30,240::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:09:30,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000291196 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:30,269::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:30,269::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:30,269::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:30,269::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) (190642) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) (190642) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) (190642) Building response >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) (190642) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:30,270::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:09:30,271::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:30,271::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:09:30,271::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:30,271::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:30,271::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) (190643) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) (190643) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) (190643) Building response >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) (190643) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:30,272::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) (190644) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) (190644) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:30,273::__init__::225::IOProcess::(_processLogs) (190644) Building response >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) (190644) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:30,274::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) (190645) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) (190645) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) (190645) Building response >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) (190645) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:30,275::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:09:30,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:30,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112652 s, 90.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:31,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:31,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:31,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:32,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:32,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114443 s, 89.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:33,767::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:09:33,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`79cf30d9-2d76-4343-9504-eaf31685a28f`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:33,768::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:09:33,768::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:33,768::task::1191::Storage.TaskManager.Task::(prepare) Task=`79cf30d9-2d76-4343-9504-eaf31685a28f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:33,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`79cf30d9-2d76-4343-9504-eaf31685a28f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:33,768::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:33,769::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:33,769::task::993::Storage.TaskManager.Task::(_decref) Task=`79cf30d9-2d76-4343-9504-eaf31685a28f`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:09:33,802::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:09:33,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f849ebd-c18d-49b4-9d03-be22d3c883c8`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:09:33,803::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:09:33,803::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f4fc6085-35e1-4f01-8e6e-af869bdfaf2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:09:33,803::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:09:33,803::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:09:33,803::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f4fc6085-35e1-4f01-8e6e-af869bdfaf2d`::Granted request >Thread-171::DEBUG::2014-08-22 09:09:33,805::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3f849ebd-c18d-49b4-9d03-be22d3c883c8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:09:33,805::task::993::Storage.TaskManager.Task::(_decref) Task=`3f849ebd-c18d-49b4-9d03-be22d3c883c8`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:09:33,807::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:33,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f849ebd-c18d-49b4-9d03-be22d3c883c8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:33,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f849ebd-c18d-49b4-9d03-be22d3c883c8`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:09:33,807::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:09:33,807::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:09:33,807::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:09:33,807::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:09:33,807::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:09:33,808::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:09:33,808::task::993::Storage.TaskManager.Task::(_decref) Task=`3f849ebd-c18d-49b4-9d03-be22d3c883c8`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:09:34,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:35,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117226 s, 87.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:36,263::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:36,263::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:36,263::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:36,263::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) (190646) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) (190646) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190646) Building response >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) (190646) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:36,264::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:09:36,265::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:09:36,293::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000309082 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:36,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:36,293::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:36,293::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) (190647) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) (190647) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) (190647) Building response >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) (190647) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:36,294::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:36,368::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:36,368::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:36,369::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:37,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:37,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117556 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:39,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:39,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011707 s, 87.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:40,276::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:40,276::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:40,276::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) (190648) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) (190648) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190648) Building response >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) (190648) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:40,277::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:09:40,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:09:40,306::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000276849 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) (190649) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:40,307::__init__::225::IOProcess::(_processLogs) (190649) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:40,308::__init__::225::IOProcess::(_processLogs) (190649) Building response >Thread-14::DEBUG::2014-08-22 09:09:40,308::__init__::225::IOProcess::(_processLogs) (190649) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:40,308::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:40,308::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) (190650) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) (190650) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:40,309::__init__::225::IOProcess::(_processLogs) (190650) Building response >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) (190650) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:40,310::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) (190651) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) (190651) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) (190651) Building response >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) (190651) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:40,311::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) (190652) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) (190652) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:40,312::__init__::225::IOProcess::(_processLogs) (190652) Building response >Thread-14::DEBUG::2014-08-22 09:09:40,313::__init__::225::IOProcess::(_processLogs) (190652) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:40,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:40,313::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:09:41,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:41,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119637 s, 85.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:41,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:41,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:41,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:43,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:43,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115788 s, 88.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:43,906::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:09:43,906::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3151caa-e39f-4490-a6ae-dd86b52c215e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:43,907::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:09:43,907::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:43,907::task::1191::Storage.TaskManager.Task::(prepare) Task=`b3151caa-e39f-4490-a6ae-dd86b52c215e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:43,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3151caa-e39f-4490-a6ae-dd86b52c215e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:43,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:43,907::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:43,907::task::993::Storage.TaskManager.Task::(_decref) Task=`b3151caa-e39f-4490-a6ae-dd86b52c215e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:09:43,958::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:09:43,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9572e93-5215-4b4e-bc1e-69a40b495747`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:09:43,958::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:09:43,958::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4227c587-2eb0-41bd-bd1c-f2b2e3b3cf72`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:09:43,958::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:09:43,959::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:09:43,959::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4227c587-2eb0-41bd-bd1c-f2b2e3b3cf72`::Granted request >Thread-171::DEBUG::2014-08-22 09:09:43,959::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d9572e93-5215-4b4e-bc1e-69a40b495747`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:09:43,959::task::993::Storage.TaskManager.Task::(_decref) Task=`d9572e93-5215-4b4e-bc1e-69a40b495747`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:09:43,961::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:43,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9572e93-5215-4b4e-bc1e-69a40b495747`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:43,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9572e93-5215-4b4e-bc1e-69a40b495747`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:09:43,961::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:09:43,961::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:09:43,962::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:09:43,962::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:09:43,962::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:09:43,962::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:09:43,962::task::993::Storage.TaskManager.Task::(_decref) Task=`d9572e93-5215-4b4e-bc1e-69a40b495747`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:09:45,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`022ed82b-969f-4b49-a02d-f749832e0b7a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:45,174::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:09:45,174::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000309082', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000276849', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:09:45,174::task::1191::Storage.TaskManager.Task::(prepare) Task=`022ed82b-969f-4b49-a02d-f749832e0b7a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000309082', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000276849', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:09:45,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`022ed82b-969f-4b49-a02d-f749832e0b7a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:45,174::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:45,174::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:45,174::task::993::Storage.TaskManager.Task::(_decref) Task=`022ed82b-969f-4b49-a02d-f749832e0b7a`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:09:45,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:45,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:45,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117579 s, 87.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) (190653) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:46,296::__init__::225::IOProcess::(_processLogs) (190653) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:46,297::__init__::225::IOProcess::(_processLogs) (190653) Building response >Thread-14::DEBUG::2014-08-22 09:09:46,297::__init__::225::IOProcess::(_processLogs) (190653) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:46,297::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:46,297::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:09:46,297::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:09:46,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000230062 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:46,326::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:46,327::__init__::225::IOProcess::(_processLogs) (190654) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:46,327::__init__::225::IOProcess::(_processLogs) (190654) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:46,327::__init__::225::IOProcess::(_processLogs) (190654) Building response >Thread-14::DEBUG::2014-08-22 09:09:46,327::__init__::225::IOProcess::(_processLogs) (190654) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:46,327::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:46,327::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:46,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:46,382::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:46,382::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:47,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:47,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018483 s, 55.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:49,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:49,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216024 s, 47.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:50,315::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:50,316::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:50,316::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:50,317::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:50,318::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:50,318::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:50,319::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:50,320::__init__::225::IOProcess::(_processLogs) (190655) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:50,320::__init__::225::IOProcess::(_processLogs) (190655) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:50,321::__init__::225::IOProcess::(_processLogs) (190655) Building response >Thread-14::DEBUG::2014-08-22 09:09:50,321::__init__::225::IOProcess::(_processLogs) (190655) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:50,322::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:50,323::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:09:50,326::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:09:50,355::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200916 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:50,358::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:50,358::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:09:50,359::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:50,360::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:50,360::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:50,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:50,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:50,362::__init__::225::IOProcess::(_processLogs) (190656) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:50,363::__init__::225::IOProcess::(_processLogs) (190656) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:50,364::__init__::225::IOProcess::(_processLogs) (190656) Building response >Thread-14::DEBUG::2014-08-22 09:09:50,364::__init__::225::IOProcess::(_processLogs) (190656) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:50,365::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:50,366::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:09:50,370::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:50,371::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:09:50,371::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:50,372::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:50,373::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:50,374::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:50,374::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:50,375::__init__::225::IOProcess::(_processLogs) (190657) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:50,376::__init__::225::IOProcess::(_processLogs) (190657) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:50,376::__init__::225::IOProcess::(_processLogs) (190657) Building response >Thread-14::DEBUG::2014-08-22 09:09:50,377::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190657) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:50,378::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:50,378::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:50,381::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:50,381::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) (190658) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) (190658) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:50,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190658) Building response >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) (190658) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:50,383::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) (190659) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) (190659) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) (190659) Building response >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) (190659) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:09:50,384::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:51,388::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:51,389::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:51,389::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:51,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:51,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118186 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:09:53,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:53,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113769 s, 90.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:09:54,008::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:09:54,009::task::595::Storage.TaskManager.Task::(_updateState) Task=`b66d9c37-a606-40da-b4d9-2ed5d80148d9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:09:54,009::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:09:54,009::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:54,009::task::1191::Storage.TaskManager.Task::(prepare) Task=`b66d9c37-a606-40da-b4d9-2ed5d80148d9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:09:54,009::task::595::Storage.TaskManager.Task::(_updateState) Task=`b66d9c37-a606-40da-b4d9-2ed5d80148d9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:09:54,009::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:09:54,010::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:09:54,010::task::993::Storage.TaskManager.Task::(_decref) Task=`b66d9c37-a606-40da-b4d9-2ed5d80148d9`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:09:54,060::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:09:54,060::task::595::Storage.TaskManager.Task::(_updateState) Task=`80524848-cd1c-4d59-9bb9-223bf793dc90`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:09:54,060::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:09:54,060::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9ea09e2b-252e-4d68-8ce2-246485c8eb36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:09:54,061::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:09:54,061::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:09:54,061::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9ea09e2b-252e-4d68-8ce2-246485c8eb36`::Granted request >Thread-171::DEBUG::2014-08-22 09:09:54,061::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`80524848-cd1c-4d59-9bb9-223bf793dc90`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:09:54,061::task::993::Storage.TaskManager.Task::(_decref) Task=`80524848-cd1c-4d59-9bb9-223bf793dc90`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:09:54,063::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:54,063::task::1191::Storage.TaskManager.Task::(prepare) Task=`80524848-cd1c-4d59-9bb9-223bf793dc90`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:09:54,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`80524848-cd1c-4d59-9bb9-223bf793dc90`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:09:54,063::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:09:54,063::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:09:54,064::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:09:54,064::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:09:54,064::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:09:54,064::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:09:54,064::task::993::Storage.TaskManager.Task::(_decref) Task=`80524848-cd1c-4d59-9bb9-223bf793dc90`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:09:55,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:55,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116966 s, 87.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:56,328::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:56,328::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) (190660) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) (190660) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) (190660) Building response >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) (190660) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:09:56,329::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:09:56,330::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:09:56,358::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000226653 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) (190661) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) (190661) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:09:56,359::__init__::225::IOProcess::(_processLogs) (190661) Building response >Thread-14::DEBUG::2014-08-22 09:09:56,360::__init__::225::IOProcess::(_processLogs) (190661) Queuing response >Thread-14::DEBUG::2014-08-22 09:09:56,360::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:09:56,360::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:56,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:56,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:09:56,396::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:09:57,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:09:58,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119007 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:00,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:00,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115843 s, 88.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:00,385::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:00,385::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) (190662) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) (190662) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) (190662) Building response >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) (190662) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:00,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:10:00,387::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:10:00,415::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223909 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:00,416::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:00,416::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:00,416::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:00,416::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:00,416::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:00,416::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) (190663) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) (190663) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) (190663) Building response >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) (190663) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:00,417::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:00,418::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) (190664) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) (190664) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) (190664) Building response >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) (190664) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:00,419::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) (190665) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) (190665) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) (190665) Building response >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) (190665) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:00,420::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:00,421::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:00,422::__init__::225::IOProcess::(_processLogs) (190666) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:00,422::__init__::225::IOProcess::(_processLogs) (190666) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:00,422::__init__::225::IOProcess::(_processLogs) (190666) Building response >Thread-14::DEBUG::2014-08-22 09:10:00,422::__init__::225::IOProcess::(_processLogs) (190666) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:00,422::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:00,422::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:10:00,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9bbcb92-d1fc-4d43-ab78-978053ef4b2d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:00,492::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:10:00,493::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000226653', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223909', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:00,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9bbcb92-d1fc-4d43-ab78-978053ef4b2d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000226653', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223909', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:00,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9bbcb92-d1fc-4d43-ab78-978053ef4b2d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:00,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:00,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:00,493::task::993::Storage.TaskManager.Task::(_decref) Task=`c9bbcb92-d1fc-4d43-ab78-978053ef4b2d`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:10:00,517::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:01,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:01,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:01,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:02,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:02,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131454 s, 77.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:04,112::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:10:04,113::task::595::Storage.TaskManager.Task::(_updateState) Task=`caeee485-e863-4c3e-876b-26f7437a0b59`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:04,113::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:10:04,113::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:04,113::task::1191::Storage.TaskManager.Task::(prepare) Task=`caeee485-e863-4c3e-876b-26f7437a0b59`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:04,113::task::595::Storage.TaskManager.Task::(_updateState) Task=`caeee485-e863-4c3e-876b-26f7437a0b59`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:04,113::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:04,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:04,113::task::993::Storage.TaskManager.Task::(_decref) Task=`caeee485-e863-4c3e-876b-26f7437a0b59`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:04,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-171::DEBUG::2014-08-22 09:10:04,188::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:10:04,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`012e425b-b583-46ed-8f43-0b18a26f89cb`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:10:04,189::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:10:04,189::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`45f88eb1-9c8b-42fb-a406-36982bc2049a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:10:04,189::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:10:04,190::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:10:04,190::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`45f88eb1-9c8b-42fb-a406-36982bc2049a`::Granted request >Thread-171::DEBUG::2014-08-22 09:10:04,190::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`012e425b-b583-46ed-8f43-0b18a26f89cb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:10:04,190::task::993::Storage.TaskManager.Task::(_decref) Task=`012e425b-b583-46ed-8f43-0b18a26f89cb`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:10:04,192::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:04,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`012e425b-b583-46ed-8f43-0b18a26f89cb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:04,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`012e425b-b583-46ed-8f43-0b18a26f89cb`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:10:04,193::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:10:04,193::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:10:04,193::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:10:04,193::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:10:04,193::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:10:04,193::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:10:04,193::task::993::Storage.TaskManager.Task::(_decref) Task=`012e425b-b583-46ed-8f43-0b18a26f89cb`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:04,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115537 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:06,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:06,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115674 s, 88.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:06,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:06,361::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:06,361::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:06,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:06,361::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) (190667) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) (190667) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190667) Building response >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) (190667) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:10:06,362::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:10:06,363::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:10:06,391::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000228364 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:06,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) (190668) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) (190668) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) (190668) Building response >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) (190668) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:06,392::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:06,393::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:06,408::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:06,408::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:06,409::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:08,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:08,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119818 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:10,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:10,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115514 s, 88.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:10,444::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:10,444::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:10,445::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:10,445::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:10,446::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:10,446::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:10,447::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:10,448::__init__::225::IOProcess::(_processLogs) (190669) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:10,448::__init__::225::IOProcess::(_processLogs) (190669) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:10,449::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190669) Building response >Thread-14::DEBUG::2014-08-22 09:10:10,449::__init__::225::IOProcess::(_processLogs) (190669) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:10,450::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:10,450::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:10:10,454::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:10:10,482::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200557 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:10,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:10,485::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:10,486::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:10,486::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:10,487::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:10,487::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:10,488::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:10,489::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(190670) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:10,489::__init__::225::IOProcess::(_processLogs) (190670) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:10,490::__init__::225::IOProcess::(_processLogs) (190670) Building response >Thread-14::DEBUG::2014-08-22 09:10:10,490::__init__::225::IOProcess::(_processLogs) (190670) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:10,491::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:10,491::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:10:10,495::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:10,496::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:10:10,496::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:10,497::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:10,498::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:10,498::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:10,499::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:10,499::__init__::225::IOProcess::(_processLogs) (190671) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:10,500::__init__::225::IOProcess::(_processLogs) (190671) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:10,500::__init__::225::IOProcess::(_processLogs) (190671) Building response >Thread-14::DEBUG::2014-08-22 09:10:10,501::__init__::225::IOProcess::(_processLogs) (190671) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:10,502::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:10,503::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:10,505::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:10,506::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:10:10,507::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:10,507::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:10,508::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:10,509::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:10,510::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:10,511::__init__::225::IOProcess::(_processLogs) DEBUG|(190672) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:10,511::__init__::225::IOProcess::(_processLogs) (190672) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:10,512::__init__::225::IOProcess::(_processLogs) (190672) Building response >Thread-14::DEBUG::2014-08-22 09:10:10,513::__init__::225::IOProcess::(_processLogs) (190672) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:10,513::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:10,514::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:10,517::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:10,518::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:10:10,518::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:10,519::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:10,520::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:10,520::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:10,521::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:10,522::__init__::225::IOProcess::(_processLogs) (190673) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:10,523::__init__::225::IOProcess::(_processLogs) (190673) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:10,523::__init__::225::IOProcess::(_processLogs) DEBUG|(190673) Building response >Thread-14::DEBUG::2014-08-22 09:10:10,524::__init__::225::IOProcess::(_processLogs) (190673) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:10,525::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:10,526::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:11,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:11,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:11,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:12,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:12,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117886 s, 86.9 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:10:12,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`50fb41e0-6273-4b87-a067-490410edb3d8`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:10:12,972::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) (190674) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:10:12,973::__init__::225::IOProcess::(_processLogs) (190674) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:10:12,974::__init__::225::IOProcess::(_processLogs) (190674) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:12,974::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) (190675) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) (190675) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) (190675) Building response >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) (190675) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:12,975::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:10:12,976::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:10:12,976::task::1191::Storage.TaskManager.Task::(prepare) Task=`50fb41e0-6273-4b87-a067-490410edb3d8`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:10:12,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`50fb41e0-6273-4b87-a067-490410edb3d8`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:10:12,976::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:10:12,976::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:10:12,976::task::993::Storage.TaskManager.Task::(_decref) Task=`50fb41e0-6273-4b87-a067-490410edb3d8`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:10:14,240::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:10:14,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`93e68975-05e4-40bf-ad1d-34aef4ede97a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:14,240::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:10:14,241::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:14,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`93e68975-05e4-40bf-ad1d-34aef4ede97a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:14,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`93e68975-05e4-40bf-ad1d-34aef4ede97a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:14,241::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:14,241::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:14,241::task::993::Storage.TaskManager.Task::(_decref) Task=`93e68975-05e4-40bf-ad1d-34aef4ede97a`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:10:14,294::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:10:14,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`383ceb42-796f-486e-b084-14d5401ab937`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:10:14,294::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:10:14,295::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`24167ade-3452-4203-8c51-232ce92dcc11`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:10:14,295::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:10:14,295::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:10:14,295::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`24167ade-3452-4203-8c51-232ce92dcc11`::Granted request >Thread-171::DEBUG::2014-08-22 09:10:14,295::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`383ceb42-796f-486e-b084-14d5401ab937`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:10:14,295::task::993::Storage.TaskManager.Task::(_decref) Task=`383ceb42-796f-486e-b084-14d5401ab937`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:10:14,297::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:14,297::task::1191::Storage.TaskManager.Task::(prepare) Task=`383ceb42-796f-486e-b084-14d5401ab937`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:14,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`383ceb42-796f-486e-b084-14d5401ab937`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:10:14,298::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:10:14,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:10:14,298::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:10:14,298::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:10:14,298::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:10:14,298::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:10:14,298::task::993::Storage.TaskManager.Task::(_decref) Task=`383ceb42-796f-486e-b084-14d5401ab937`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:14,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:14,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117024 s, 87.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:15,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3b5480c-e157-4e6b-b9f1-ff6f24e78578`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:15,689::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:10:15,689::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000228364', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000200557', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:15,689::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3b5480c-e157-4e6b-b9f1-ff6f24e78578`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000228364', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000200557', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:15,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3b5480c-e157-4e6b-b9f1-ff6f24e78578`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:15,689::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:15,689::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:15,689::task::993::Storage.TaskManager.Task::(_decref) Task=`e3b5480c-e157-4e6b-b9f1-ff6f24e78578`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:10:15,715::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) (190676) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) (190676) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:16,394::__init__::225::IOProcess::(_processLogs) (190676) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:16,395::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:16,395::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:10:16,395::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:16,424::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:10:16,424::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000256655 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:16,426::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:16,427::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:16,427::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:16,428::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:16,428::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:16,429::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:16,429::__init__::225::IOProcess::(_processLogs) (190677) Finding callback 'statvfs'... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:16,429::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:16,430::__init__::225::IOProcess::(_processLogs) (190677) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:16,430::__init__::225::IOProcess::(_processLogs) (190677) Building response >Thread-14::DEBUG::2014-08-22 09:10:16,431::__init__::225::IOProcess::(_processLogs) (190677) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:16,431::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:16,431::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:10:16,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:16,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118633 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:18,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:18,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124752 s, 82.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:20,528::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) (190678) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) (190678) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) (190678) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:20,529::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:10:20,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:10:20,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000238011 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) (190679) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:20,559::__init__::225::IOProcess::(_processLogs) (190679) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:20,560::__init__::225::IOProcess::(_processLogs) (190679) Building response >Thread-14::DEBUG::2014-08-22 09:10:20,560::__init__::225::IOProcess::(_processLogs) (190679) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:20,560::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:20,560::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) (190680) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) (190680) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) (190680) Building response >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) (190680) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:20,561::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:20,562::__init__::225::IOProcess::(_processLogs) (190681) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) (190681) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) (190681) Building response >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) (190681) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:10:20,563::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) (190682) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) (190682) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) (190682) Building response >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) (190682) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:20,564::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:10:20,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:20,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119887 s, 85.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:21,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:21,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:21,438::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:22,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:22,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011787 s, 86.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:24,354::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:10:24,354::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ae3430e-3688-4334-8a2d-a99f82ad9449`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:24,355::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:10:24,355::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:24,355::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ae3430e-3688-4334-8a2d-a99f82ad9449`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:24,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ae3430e-3688-4334-8a2d-a99f82ad9449`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:24,355::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:24,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:24,355::task::993::Storage.TaskManager.Task::(_decref) Task=`1ae3430e-3688-4334-8a2d-a99f82ad9449`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:10:24,387::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:10:24,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`83bbab6e-4fe9-4475-9fe2-003dba9f10ed`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:10:24,388::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:10:24,388::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f8aca0df-8543-44e1-9642-4a26e8b67763`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:10:24,388::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:10:24,388::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:10:24,388::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f8aca0df-8543-44e1-9642-4a26e8b67763`::Granted request >Thread-171::DEBUG::2014-08-22 09:10:24,389::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`83bbab6e-4fe9-4475-9fe2-003dba9f10ed`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:10:24,389::task::993::Storage.TaskManager.Task::(_decref) Task=`83bbab6e-4fe9-4475-9fe2-003dba9f10ed`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:10:24,391::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:24,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`83bbab6e-4fe9-4475-9fe2-003dba9f10ed`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:24,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`83bbab6e-4fe9-4475-9fe2-003dba9f10ed`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:10:24,391::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:10:24,391::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:10:24,391::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:10:24,391::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:10:24,391::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:10:24,391::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:10:24,391::task::993::Storage.TaskManager.Task::(_decref) Task=`83bbab6e-4fe9-4475-9fe2-003dba9f10ed`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:24,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:25,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121532 s, 84.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) (190683) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) (190683) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) (190683) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:26,433::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:26,434::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:10:26,434::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:26,463::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:10:26,463::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000171041 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:26,465::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:26,466::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:26,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:26,467::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:26,467::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:26,468::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:26,468::__init__::225::IOProcess::(_processLogs) (190684) Finding callback 'statvfs'... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:26,469::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:26,469::__init__::225::IOProcess::(_processLogs) (190684) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:26,469::__init__::225::IOProcess::(_processLogs) (190684) Building response >Thread-14::DEBUG::2014-08-22 09:10:26,470::__init__::225::IOProcess::(_processLogs) (190684) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:26,470::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:26,470::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:10:27,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:27,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119426 s, 85.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:29,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:29,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117731 s, 87.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:30,565::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:30,565::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) (190685) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) (190685) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) (190685) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:30,566::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:10:30,567::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:10:30,595::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00022771 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) (190686) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) (190686) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) (190686) Building response >Thread-14::DEBUG::2014-08-22 09:10:30,596::__init__::225::IOProcess::(_processLogs) (190686) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:30,597::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:10:30,597::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:10:30,597::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) (190687) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) (190687) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) (190687) Building response >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) (190687) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:30,598::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) (190688) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) (190688) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) (190688) Building response >Thread-14::DEBUG::2014-08-22 09:10:30,599::__init__::225::IOProcess::(_processLogs) (190688) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:30,600::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:30,601::__init__::225::IOProcess::(_processLogs) (190689) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:30,601::__init__::225::IOProcess::(_processLogs) (190689) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:30,601::__init__::225::IOProcess::(_processLogs) (190689) Building response >Thread-14::DEBUG::2014-08-22 09:10:30,601::__init__::225::IOProcess::(_processLogs) (190689) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:30,601::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:30,601::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:10:30,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`0950886b-0908-4c0e-874e-298c365475a1`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:30,986::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:10:30,986::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000171041', 'lastCheck': '4.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00022771', 'lastCheck': '0.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:30,986::task::1191::Storage.TaskManager.Task::(prepare) Task=`0950886b-0908-4c0e-874e-298c365475a1`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000171041', 'lastCheck': '4.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00022771', 'lastCheck': '0.4', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:30,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`0950886b-0908-4c0e-874e-298c365475a1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:30,986::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:30,986::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:30,986::task::993::Storage.TaskManager.Task::(_decref) Task=`0950886b-0908-4c0e-874e-298c365475a1`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:10:31,012::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:31,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:31,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117619 s, 87.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:31,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:31,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:31,477::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:33,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:33,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120472 s, 85.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:34,440::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:10:34,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`d09728e6-4635-42bf-ba6b-78bf247ccacd`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:34,440::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:10:34,440::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:34,440::task::1191::Storage.TaskManager.Task::(prepare) Task=`d09728e6-4635-42bf-ba6b-78bf247ccacd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:34,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`d09728e6-4635-42bf-ba6b-78bf247ccacd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:34,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:34,441::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:34,441::task::993::Storage.TaskManager.Task::(_decref) Task=`d09728e6-4635-42bf-ba6b-78bf247ccacd`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:10:34,482::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:10:34,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fcf9244-30dd-433c-9b50-848c24302fa4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:10:34,483::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:10:34,483::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`34c8e412-ce56-45ef-9073-27e0934bf6eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:10:34,483::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:10:34,483::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:10:34,483::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`34c8e412-ce56-45ef-9073-27e0934bf6eb`::Granted request >Thread-171::DEBUG::2014-08-22 09:10:34,484::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0fcf9244-30dd-433c-9b50-848c24302fa4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:10:34,484::task::993::Storage.TaskManager.Task::(_decref) Task=`0fcf9244-30dd-433c-9b50-848c24302fa4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:10:34,486::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:34,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`0fcf9244-30dd-433c-9b50-848c24302fa4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:34,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fcf9244-30dd-433c-9b50-848c24302fa4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:10:34,486::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:10:34,486::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:10:34,486::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:10:34,486::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:10:34,486::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:10:34,486::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:10:34,487::task::993::Storage.TaskManager.Task::(_decref) Task=`0fcf9244-30dd-433c-9b50-848c24302fa4`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:35,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:35,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123502 s, 82.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) (190690) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) (190690) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:36,472::__init__::225::IOProcess::(_processLogs) (190690) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:36,473::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:36,473::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:10:36,473::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:36,503::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:10:36,503::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000180243 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:36,505::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:36,506::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:36,506::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:36,506::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:36,507::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:36,507::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:36,508::__init__::225::IOProcess::(_processLogs) (190691) Finding callback 'statvfs'... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:36,508::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:36,509::__init__::225::IOProcess::(_processLogs) (190691) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:36,509::__init__::225::IOProcess::(_processLogs) (190691) Building response >Thread-14::DEBUG::2014-08-22 09:10:36,510::__init__::225::IOProcess::(_processLogs) (190691) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:36,510::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:36,510::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:10:37,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:37,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120777 s, 84.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:39,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:39,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118437 s, 86.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:40,602::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:40,602::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:40,602::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:40,602::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:40,602::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:40,603::__init__::225::IOProcess::(_processLogs) (190692) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:40,603::__init__::225::IOProcess::(_processLogs) (190692) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:40,603::__init__::225::IOProcess::(_processLogs) (190692) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:40,603::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:40,603::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:10:40,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:10:40,632::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000306825 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:40,632::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:40,632::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:40,632::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) (190693) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) (190693) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) (190693) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:40,633::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) (190694) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:40,634::__init__::225::IOProcess::(_processLogs) (190694) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) (190694) Building response >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) (190694) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:40,635::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) (190695) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) (190695) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) (190695) Building response >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) (190695) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:40,636::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) (190696) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) (190696) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) (190696) Building response >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) (190696) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:40,637::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:41,516::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:41,516::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:41,516::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:41,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:41,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117255 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:43,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:43,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115815 s, 88.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:44,573::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:10:44,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcd46ee9-1ea4-4367-bf1f-e28c611d6250`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:44,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:10:44,574::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:44,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`dcd46ee9-1ea4-4367-bf1f-e28c611d6250`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:44,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcd46ee9-1ea4-4367-bf1f-e28c611d6250`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:44,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:44,574::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:44,575::task::993::Storage.TaskManager.Task::(_decref) Task=`dcd46ee9-1ea4-4367-bf1f-e28c611d6250`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:10:44,620::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:10:44,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eca25e0-f547-46ee-a1af-ef8d689c5212`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:10:44,621::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:10:44,621::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6d229476-52a6-40b6-aa69-3f1135c85e0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:10:44,621::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:10:44,621::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:10:44,621::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6d229476-52a6-40b6-aa69-3f1135c85e0f`::Granted request >Thread-171::DEBUG::2014-08-22 09:10:44,622::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7eca25e0-f547-46ee-a1af-ef8d689c5212`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:10:44,622::task::993::Storage.TaskManager.Task::(_decref) Task=`7eca25e0-f547-46ee-a1af-ef8d689c5212`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:10:44,624::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:44,624::task::1191::Storage.TaskManager.Task::(prepare) Task=`7eca25e0-f547-46ee-a1af-ef8d689c5212`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108921880576', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:44,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eca25e0-f547-46ee-a1af-ef8d689c5212`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:10:44,624::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:10:44,624::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:10:44,624::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:10:44,625::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:10:44,625::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:10:44,625::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:10:44,625::task::993::Storage.TaskManager.Task::(_decref) Task=`7eca25e0-f547-46ee-a1af-ef8d689c5212`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:45,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:45,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114271 s, 89.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:46,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b1eaf7-1c5c-4342-b096-1248877011b0`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:46,196::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:10:46,196::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180243', 'lastCheck': '9.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000306825', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:46,197::task::1191::Storage.TaskManager.Task::(prepare) Task=`68b1eaf7-1c5c-4342-b096-1248877011b0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180243', 'lastCheck': '9.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000306825', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:10:46,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b1eaf7-1c5c-4342-b096-1248877011b0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:46,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:46,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:46,197::task::993::Storage.TaskManager.Task::(_decref) Task=`68b1eaf7-1c5c-4342-b096-1248877011b0`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:10:46,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:46,511::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) (190697) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) (190697) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) (190697) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:46,512::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:10:46,513::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:46,542::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:10:46,542::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199816 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:46,544::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:46,544::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:46,545::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:46,545::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:46,546::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:46,546::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:46,547::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:46,547::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:46,548::__init__::225::IOProcess::(_processLogs) (190698) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:46,548::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:46,548::__init__::225::IOProcess::(_processLogs) (190698) Building response >Thread-14::DEBUG::2014-08-22 09:10:46,549::__init__::225::IOProcess::(_processLogs) (190698) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:46,549::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:46,549::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:10:47,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:47,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012038 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:49,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:49,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240547 s, 42.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:50,639::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:50,639::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:50,639::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:50,639::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:50,639::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) (190699) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) (190699) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) (190699) Building response >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) (190699) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:50,640::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:10:50,641::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:10:50,669::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000233575 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) (190700) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) (190700) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:50,670::__init__::225::IOProcess::(_processLogs) (190700) Building response >Thread-14::DEBUG::2014-08-22 09:10:50,671::__init__::225::IOProcess::(_processLogs) (190700) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:50,671::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:50,671::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:10:50,671::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) (190701) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) (190701) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) (190701) Building response >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) (190701) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:50,672::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:50,673::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) (190702) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) (190702) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) (190702) Building response >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) (190702) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:50,674::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) (190703) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) (190703) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) (190703) Building response >Thread-14::DEBUG::2014-08-22 09:10:50,675::__init__::225::IOProcess::(_processLogs) (190703) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:50,676::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:50,676::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:51,554::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:51,554::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:51,555::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:10:52,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:52,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0151309 s, 67.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:10:54,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:54,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011937 s, 85.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:10:54,697::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3e0a2e8c] >Thread-13::DEBUG::2014-08-22 09:10:54,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`2dac9b16-9180-47a7-bbcb-0f7636784d8a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:10:54,697::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:10:54,698::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:54,698::task::1191::Storage.TaskManager.Task::(prepare) Task=`2dac9b16-9180-47a7-bbcb-0f7636784d8a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:10:54,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`2dac9b16-9180-47a7-bbcb-0f7636784d8a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:10:54,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:10:54,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:10:54,698::task::993::Storage.TaskManager.Task::(_decref) Task=`2dac9b16-9180-47a7-bbcb-0f7636784d8a`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:10:54,748::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3e0a2e8c] >Thread-171::DEBUG::2014-08-22 09:10:54,748::task::595::Storage.TaskManager.Task::(_updateState) Task=`738f3d5f-1d55-4aaf-a18c-9fa575097497`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:10:54,748::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:10:54,748::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`61cef7ac-cddb-45b3-80ae-a5ec3f4cc06d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:10:54,748::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:10:54,749::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:10:54,749::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`61cef7ac-cddb-45b3-80ae-a5ec3f4cc06d`::Granted request >Thread-171::DEBUG::2014-08-22 09:10:54,749::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`738f3d5f-1d55-4aaf-a18c-9fa575097497`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:10:54,749::task::993::Storage.TaskManager.Task::(_decref) Task=`738f3d5f-1d55-4aaf-a18c-9fa575097497`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:10:54,751::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:54,751::task::1191::Storage.TaskManager.Task::(prepare) Task=`738f3d5f-1d55-4aaf-a18c-9fa575097497`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:10:54,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`738f3d5f-1d55-4aaf-a18c-9fa575097497`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:10:54,751::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:10:54,751::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:10:54,752::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:10:54,752::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:10:54,752::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:10:54,752::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:10:54,752::task::993::Storage.TaskManager.Task::(_decref) Task=`738f3d5f-1d55-4aaf-a18c-9fa575097497`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:10:56,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:56,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115388 s, 88.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:10:56,550::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:56,550::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:56,550::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:10:56,550::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) (190704) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) (190704) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) (190704) Building response >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) (190704) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:56,551::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:10:56,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:10:56,581::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199258 s, 2.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:56,581::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:56,583::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:10:56,583::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:10:56,584::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:56,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:56,585::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:10:56,585::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:10:56,586::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:10:56,586::__init__::225::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:10:56,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:10:56,587::__init__::225::IOProcess::(_processLogs) (190705) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:10:56,587::__init__::225::IOProcess::(_processLogs) (190705) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:10:56,588::__init__::225::IOProcess::(_processLogs) (190705) Building response >Thread-14::DEBUG::2014-08-22 09:10:56,588::__init__::225::IOProcess::(_processLogs) (190705) Queuing response >Thread-14::DEBUG::2014-08-22 09:10:56,588::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:10:56,589::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:10:58,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:10:58,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121657 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:00,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:00,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117673 s, 87.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:00,677::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:00,677::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:00,677::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:00,677::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) (190706) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) (190706) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) (190706) Building response >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) (190706) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:00,678::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:11:00,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:11:00,707::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000255589 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:00,708::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:00,709::__init__::225::IOProcess::(_processLogs) (190707) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:00,709::__init__::225::IOProcess::(_processLogs) (190707) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:00,709::__init__::225::IOProcess::(_processLogs) (190707) Building response >Thread-14::DEBUG::2014-08-22 09:11:00,709::__init__::225::IOProcess::(_processLogs) (190707) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:00,709::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:00,709::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) (190708) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:00,710::__init__::225::IOProcess::(_processLogs) (190708) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) (190708) Building response >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) (190708) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:11:00,711::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) (190709) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) (190709) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) (190709) Building response >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) (190709) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:00,712::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:00,713::__init__::225::IOProcess::(_processLogs) (190710) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:00,714::__init__::225::IOProcess::(_processLogs) (190710) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:00,714::__init__::225::IOProcess::(_processLogs) (190710) Building response >Thread-14::DEBUG::2014-08-22 09:11:00,714::__init__::225::IOProcess::(_processLogs) (190710) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:00,714::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:00,714::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:11:01,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d88b4c5-dc7e-473e-8260-cea18975ba69`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:01,423::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:11:01,423::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199258', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000255589', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:01,423::task::1191::Storage.TaskManager.Task::(prepare) Task=`5d88b4c5-dc7e-473e-8260-cea18975ba69`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199258', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000255589', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:01,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d88b4c5-dc7e-473e-8260-cea18975ba69`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:01,423::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:01,423::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:01,424::task::993::Storage.TaskManager.Task::(_decref) Task=`5d88b4c5-dc7e-473e-8260-cea18975ba69`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:11:01,445::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:01,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:01,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:01,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:02,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:02,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119104 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:04,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:04,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114464 s, 89.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:11:04,817::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:11:04,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`a55e1096-e39c-438b-8765-037852879571`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:04,817::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:11:04,818::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:04,818::task::1191::Storage.TaskManager.Task::(prepare) Task=`a55e1096-e39c-438b-8765-037852879571`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:04,818::task::595::Storage.TaskManager.Task::(_updateState) Task=`a55e1096-e39c-438b-8765-037852879571`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:04,818::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:04,818::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:04,818::task::993::Storage.TaskManager.Task::(_decref) Task=`a55e1096-e39c-438b-8765-037852879571`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:11:04,841::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:11:04,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9ed48c8-6fb1-4daf-9712-fd4e661542f6`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:11:04,842::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:11:04,842::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e8d46ba1-99f6-4366-a865-b0ff115cf80f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:11:04,842::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:11:04,842::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:11:04,842::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e8d46ba1-99f6-4366-a865-b0ff115cf80f`::Granted request >Thread-171::DEBUG::2014-08-22 09:11:04,842::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c9ed48c8-6fb1-4daf-9712-fd4e661542f6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:11:04,843::task::993::Storage.TaskManager.Task::(_decref) Task=`c9ed48c8-6fb1-4daf-9712-fd4e661542f6`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:11:04,845::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:04,845::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9ed48c8-6fb1-4daf-9712-fd4e661542f6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:04,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9ed48c8-6fb1-4daf-9712-fd4e661542f6`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:11:04,845::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:11:04,845::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:11:04,845::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:11:04,845::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:11:04,845::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:11:04,845::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:11:04,846::task::993::Storage.TaskManager.Task::(_decref) Task=`c9ed48c8-6fb1-4daf-9712-fd4e661542f6`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:06,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-22 09:11:06,590::__init__::225::IOProcess::(_processLogs) (190698) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:06,590::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:06,590::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:06,590::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:06,591::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:06,591::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:06,591::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:06,591::__init__::225::IOProcess::(_processLogs) (190711) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:06,591::__init__::225::IOProcess::(_processLogs) (190711) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:06,596::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190711) Building response >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:06,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:11:06,597::__init__::225::IOProcess::(_processLogs) (190711) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:06,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:06,600::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:06,601::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:11:06,604::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:06,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:11:06,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000230182 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:06,638::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:06,639::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:06,639::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:06,640::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:06,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:06,641::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:06,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:06,642::__init__::225::IOProcess::(_processLogs) (190712) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:06,643::__init__::225::IOProcess::(_processLogs) (190712) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:06,670::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190712) Building response >Dummy-170::DEBUG::2014-08-22 09:11:06,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0569978 s, 18.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:06,671::__init__::225::IOProcess::(_processLogs) (190712) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:06,672::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:06,672::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:11:08,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:08,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018246 s, 56.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:10,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:10,715::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) (190713) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:10,717::__init__::225::IOProcess::(_processLogs) (190713) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190713) Building response >Thread-14::DEBUG::2014-08-22 09:11:10,720::__init__::225::IOProcess::(_processLogs) (190713) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:11:10,721::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:11:10,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:11:10,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000268443 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:10,750::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:10,751::__init__::225::IOProcess::(_processLogs) (190714) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:10,751::__init__::225::IOProcess::(_processLogs) (190714) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:10,751::__init__::225::IOProcess::(_processLogs) (190714) Building response >Thread-14::DEBUG::2014-08-22 09:11:10,751::__init__::225::IOProcess::(_processLogs) (190714) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:10,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:10,751::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:10,752::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) (190715) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) (190715) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) (190715) Building response >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) (190715) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:10,753::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) (190716) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) (190716) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) (190716) Building response >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) (190716) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:10,754::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:10,755::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) (190717) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) (190717) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) (190717) Building response >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) (190717) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:10,756::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:11:10,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:10,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118207 s, 86.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:11,643::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:11,643::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:11,644::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:12,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:12,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117598 s, 87.1 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:11:13,421::task::595::Storage.TaskManager.Task::(_updateState) Task=`4064f21a-689e-4061-942d-5e727b424704`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:11:13,421::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) (190718) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:11:13,422::__init__::225::IOProcess::(_processLogs) (190718) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:11:13,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190718) Building response >Thread-14::DEBUG::2014-08-22 09:11:13,423::__init__::225::IOProcess::(_processLogs) (190718) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:13,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:11:13,423::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) (190719) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:11:13,424::__init__::225::IOProcess::(_processLogs) (190719) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:11:13,425::__init__::225::IOProcess::(_processLogs) (190719) Building response >Thread-14::DEBUG::2014-08-22 09:11:13,425::__init__::225::IOProcess::(_processLogs) (190719) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:13,425::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:13,425::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:11:13,425::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:11:13,425::task::1191::Storage.TaskManager.Task::(prepare) Task=`4064f21a-689e-4061-942d-5e727b424704`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:11:13,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`4064f21a-689e-4061-942d-5e727b424704`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:11:13,426::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:11:13,426::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:11:13,426::task::993::Storage.TaskManager.Task::(_decref) Task=`4064f21a-689e-4061-942d-5e727b424704`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:11:14,892::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:11:14,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`65794c23-bb46-442a-948a-9ef47ef6d8bf`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:14,892::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:11:14,892::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:14,893::task::1191::Storage.TaskManager.Task::(prepare) Task=`65794c23-bb46-442a-948a-9ef47ef6d8bf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:14,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`65794c23-bb46-442a-948a-9ef47ef6d8bf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:14,893::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:14,893::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:14,893::task::993::Storage.TaskManager.Task::(_decref) Task=`65794c23-bb46-442a-948a-9ef47ef6d8bf`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:14,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-171::DEBUG::2014-08-22 09:11:14,940::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:11:14,941::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2061941-9bdc-48cd-a096-adf85212972b`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:11:14,941::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:11:14,941::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d18e15c2-c6fc-463d-89f8-52a8d3c76868`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:11:14,941::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:11:14,941::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:11:14,942::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d18e15c2-c6fc-463d-89f8-52a8d3c76868`::Granted request >Thread-171::DEBUG::2014-08-22 09:11:14,942::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a2061941-9bdc-48cd-a096-adf85212972b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:11:14,942::task::993::Storage.TaskManager.Task::(_decref) Task=`a2061941-9bdc-48cd-a096-adf85212972b`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:11:14,944::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:14,944::task::1191::Storage.TaskManager.Task::(prepare) Task=`a2061941-9bdc-48cd-a096-adf85212972b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:14,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2061941-9bdc-48cd-a096-adf85212972b`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:11:14,944::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:11:14,945::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:11:14,945::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:11:14,945::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:11:14,945::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:11:14,945::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:11:14,945::task::993::Storage.TaskManager.Task::(_decref) Task=`a2061941-9bdc-48cd-a096-adf85212972b`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:14,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116222 s, 88.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:11:16,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c7d6e07-d9be-46e2-a1e0-9db099600bc8`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:16,615::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:11:16,616::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000230182', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000268443', 'lastCheck': '5.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:16,616::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c7d6e07-d9be-46e2-a1e0-9db099600bc8`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000230182', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000268443', 'lastCheck': '5.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:16,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c7d6e07-d9be-46e2-a1e0-9db099600bc8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:16,616::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:16,616::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:16,616::task::993::Storage.TaskManager.Task::(_decref) Task=`4c7d6e07-d9be-46e2-a1e0-9db099600bc8`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:11:16,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:16,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:16,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:16,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:11:16,676::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:16,676::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:16,676::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:16,676::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:16,676::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) (190720) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) (190720) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) (190720) Building response >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190720) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:16,677::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:11:16,678::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:11:16,706::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000256085 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) (190721) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) (190721) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:16,707::__init__::225::IOProcess::(_processLogs) (190721) Building response >Thread-14::DEBUG::2014-08-22 09:11:16,708::__init__::225::IOProcess::(_processLogs) (190721) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:16,708::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:16,708::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:11:16,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:17,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118562 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:19,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:19,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117766 s, 87.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:20,757::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:20,757::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:20,757::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:20,757::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:20,757::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) (190722) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) (190722) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) (190722) Building response >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) (190722) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:20,758::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:11:20,759::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:11:20,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00025605 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) (190723) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:20,788::__init__::225::IOProcess::(_processLogs) (190723) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:20,789::__init__::225::IOProcess::(_processLogs) (190723) Building response >Thread-14::DEBUG::2014-08-22 09:11:20,789::__init__::225::IOProcess::(_processLogs) (190723) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:20,789::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:20,789::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) (190724) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) (190724) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:20,790::__init__::225::IOProcess::(_processLogs) (190724) Building response >Thread-14::DEBUG::2014-08-22 09:11:20,791::__init__::225::IOProcess::(_processLogs) (190724) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:20,791::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:20,791::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:20,791::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:20,791::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:11:20,791::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) (190725) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) (190725) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) (190725) Building response >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) (190725) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:20,792::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) (190726) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:20,793::__init__::225::IOProcess::(_processLogs) (190726) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:20,794::__init__::225::IOProcess::(_processLogs) (190726) Building response >Thread-14::DEBUG::2014-08-22 09:11:20,794::__init__::225::IOProcess::(_processLogs) (190726) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:20,794::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:20,794::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:11:21,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:21,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119172 s, 85.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:21,656::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:21,656::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:21,657::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:23,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:23,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116528 s, 87.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:11:24,995::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:11:24,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e9fdcb5-1738-47fd-918e-e326e5428710`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:24,995::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:11:24,996::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:24,996::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e9fdcb5-1738-47fd-918e-e326e5428710`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:24,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e9fdcb5-1738-47fd-918e-e326e5428710`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:24,996::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:24,996::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:24,996::task::993::Storage.TaskManager.Task::(_decref) Task=`9e9fdcb5-1738-47fd-918e-e326e5428710`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:11:25,033::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:11:25,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`612a546f-639c-4798-b404-e6fd5a80add6`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:11:25,033::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:11:25,034::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`08c63e8c-a0c8-4369-9123-1adc7cdd2946`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:11:25,034::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:11:25,034::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:11:25,034::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`08c63e8c-a0c8-4369-9123-1adc7cdd2946`::Granted request >Thread-171::DEBUG::2014-08-22 09:11:25,034::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`612a546f-639c-4798-b404-e6fd5a80add6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:11:25,034::task::993::Storage.TaskManager.Task::(_decref) Task=`612a546f-639c-4798-b404-e6fd5a80add6`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:11:25,036::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:25,036::task::1191::Storage.TaskManager.Task::(prepare) Task=`612a546f-639c-4798-b404-e6fd5a80add6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:25,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`612a546f-639c-4798-b404-e6fd5a80add6`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:11:25,037::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:11:25,037::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:11:25,037::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:11:25,037::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:11:25,037::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:11:25,037::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:11:25,037::task::993::Storage.TaskManager.Task::(_decref) Task=`612a546f-639c-4798-b404-e6fd5a80add6`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:25,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:25,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118011 s, 86.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:26,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:26,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:26,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:11:26,709::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:26,709::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:26,709::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:26,709::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:26,709::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:26,709::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) (190727) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) (190727) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) (190727) Building response >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) (190727) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:26,710::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:11:26,711::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:11:26,739::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000237461 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:26,739::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) (190728) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) (190728) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) (190728) Building response >Thread-14::DEBUG::2014-08-22 09:11:26,740::__init__::225::IOProcess::(_processLogs) (190728) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:26,741::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:26,741::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:11:27,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:27,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116153 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:29,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:29,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154036 s, 66.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:30,795::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:30,796::__init__::225::IOProcess::(_processLogs) (190729) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:30,796::__init__::225::IOProcess::(_processLogs) (190729) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:30,796::__init__::225::IOProcess::(_processLogs) (190729) Building response >Thread-14::DEBUG::2014-08-22 09:11:30,796::__init__::225::IOProcess::(_processLogs) (190729) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:30,796::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:30,796::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:11:30,797::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:11:30,825::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234615 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:30,825::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:30,825::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) (190730) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) (190730) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) (190730) Building response >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) (190730) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:30,826::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:11:30,827::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:30,827::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:11:30,827::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) (190731) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) (190731) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) (190731) Building response >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) (190731) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:30,828::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:30,829::__init__::225::IOProcess::(_processLogs) (190732) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) (190732) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) (190732) Building response >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) (190732) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:30,830::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) (190733) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) (190733) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) (190733) Building response >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) (190733) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:30,831::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:11:31,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:31,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122527 s, 83.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:31,669::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:31,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:31,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:11:31,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`57645f22-13eb-40ce-b511-e6386ff98966`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:31,793::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:11:31,793::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000237461', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234615', 'lastCheck': '1.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:31,793::task::1191::Storage.TaskManager.Task::(prepare) Task=`57645f22-13eb-40ce-b511-e6386ff98966`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000237461', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234615', 'lastCheck': '1.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:31,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`57645f22-13eb-40ce-b511-e6386ff98966`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:31,793::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:31,793::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:31,794::task::993::Storage.TaskManager.Task::(_decref) Task=`57645f22-13eb-40ce-b511-e6386ff98966`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:11:31,817::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:33,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:33,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117546 s, 87.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:11:35,085::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:11:35,085::task::595::Storage.TaskManager.Task::(_updateState) Task=`c92f85a4-e702-4fab-829a-a4012154edec`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:35,086::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:11:35,086::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:35,086::task::1191::Storage.TaskManager.Task::(prepare) Task=`c92f85a4-e702-4fab-829a-a4012154edec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:35,086::task::595::Storage.TaskManager.Task::(_updateState) Task=`c92f85a4-e702-4fab-829a-a4012154edec`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:35,086::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:35,086::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:35,086::task::993::Storage.TaskManager.Task::(_decref) Task=`c92f85a4-e702-4fab-829a-a4012154edec`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:11:35,120::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:11:35,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`a73fed4b-1aa2-4aad-ac46-b8a7ab5e0e3e`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:11:35,120::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:11:35,120::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`be88efec-ba10-4027-913b-8e5658075fb6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:11:35,121::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:11:35,121::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:11:35,121::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`be88efec-ba10-4027-913b-8e5658075fb6`::Granted request >Thread-171::DEBUG::2014-08-22 09:11:35,121::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a73fed4b-1aa2-4aad-ac46-b8a7ab5e0e3e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:11:35,121::task::993::Storage.TaskManager.Task::(_decref) Task=`a73fed4b-1aa2-4aad-ac46-b8a7ab5e0e3e`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:11:35,123::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:35,123::task::1191::Storage.TaskManager.Task::(prepare) Task=`a73fed4b-1aa2-4aad-ac46-b8a7ab5e0e3e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:35,123::task::595::Storage.TaskManager.Task::(_updateState) Task=`a73fed4b-1aa2-4aad-ac46-b8a7ab5e0e3e`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:11:35,123::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:11:35,124::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:11:35,124::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:11:35,124::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:11:35,124::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:11:35,124::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:11:35,124::task::993::Storage.TaskManager.Task::(_decref) Task=`a73fed4b-1aa2-4aad-ac46-b8a7ab5e0e3e`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:35,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:35,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201394 s, 50.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:36,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:36,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:36,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:36,742::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:36,743::__init__::225::IOProcess::(_processLogs) (190734) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:36,743::__init__::225::IOProcess::(_processLogs) (190734) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:36,743::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190734) Building response >Thread-14::DEBUG::2014-08-22 09:11:36,743::__init__::225::IOProcess::(_processLogs) (190734) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:36,743::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:36,743::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:11:36,744::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:11:36,772::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000230951 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:36,772::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:36,772::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) (190735) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) (190735) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) (190735) Building response >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) (190735) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:36,773::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:11:37,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:37,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119679 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:39,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:39,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118446 s, 86.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:40,832::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) (190736) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) (190736) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) (190736) Building response >Thread-14::DEBUG::2014-08-22 09:11:40,833::__init__::225::IOProcess::(_processLogs) (190736) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:40,834::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:40,834::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:11:40,834::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:11:40,862::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000264877 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:40,863::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:40,863::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:40,863::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:40,863::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) (190737) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) (190737) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) (190737) Building response >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) (190737) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:40,864::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:11:40,865::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:40,865::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:11:40,865::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:40,865::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) (190738) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) (190738) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) (190738) Building response >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) (190738) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:40,866::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) (190739) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:40,867::__init__::225::IOProcess::(_processLogs) (190739) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) (190739) Building response >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) (190739) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:11:40,868::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) (190740) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) (190740) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) (190740) Building response >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) (190740) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:40,869::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:41,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:41,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:41,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:41,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:41,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119911 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:43,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:44,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117032 s, 87.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:11:45,173::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:11:45,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`837688d7-18e7-444a-bc21-3fdaddccbb23`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:45,173::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:11:45,174::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:45,174::task::1191::Storage.TaskManager.Task::(prepare) Task=`837688d7-18e7-444a-bc21-3fdaddccbb23`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:45,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`837688d7-18e7-444a-bc21-3fdaddccbb23`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:45,174::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:45,174::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:45,174::task::993::Storage.TaskManager.Task::(_decref) Task=`837688d7-18e7-444a-bc21-3fdaddccbb23`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:11:45,244::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:11:45,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`42f385d6-0982-457a-ae7e-47b0502e1664`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:11:45,244::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:11:45,244::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4d7b9331-fead-43a5-9ba8-7864ad66472a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:11:45,244::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:11:45,245::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:11:45,245::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4d7b9331-fead-43a5-9ba8-7864ad66472a`::Granted request >Thread-171::DEBUG::2014-08-22 09:11:45,245::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`42f385d6-0982-457a-ae7e-47b0502e1664`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:11:45,245::task::993::Storage.TaskManager.Task::(_decref) Task=`42f385d6-0982-457a-ae7e-47b0502e1664`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:11:45,247::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:45,247::task::1191::Storage.TaskManager.Task::(prepare) Task=`42f385d6-0982-457a-ae7e-47b0502e1664`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:45,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`42f385d6-0982-457a-ae7e-47b0502e1664`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:11:45,247::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:11:45,248::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:11:45,248::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:11:45,248::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:11:45,248::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:11:45,248::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:11:45,248::task::993::Storage.TaskManager.Task::(_decref) Task=`42f385d6-0982-457a-ae7e-47b0502e1664`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:46,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:46,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228955 s, 44.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:46,689::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:46,689::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:46,689::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) (190741) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:46,775::__init__::225::IOProcess::(_processLogs) (190741) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:46,776::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190741) Building response >Thread-14::DEBUG::2014-08-22 09:11:46,776::__init__::225::IOProcess::(_processLogs) (190741) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:46,776::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:46,776::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:11:46,776::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:11:46,805::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000274877 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:46,805::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:46,805::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) (190742) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) (190742) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) (190742) Building response >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) (190742) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:46,806::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:11:47,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7e9f4ec-1470-48ef-937c-653789e97e4c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:47,042::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:11:47,043::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000274877', 'lastCheck': '0.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264877', 'lastCheck': '6.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:47,043::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7e9f4ec-1470-48ef-937c-653789e97e4c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000274877', 'lastCheck': '0.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264877', 'lastCheck': '6.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:11:47,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7e9f4ec-1470-48ef-937c-653789e97e4c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:47,043::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:47,043::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:47,043::task::993::Storage.TaskManager.Task::(_decref) Task=`b7e9f4ec-1470-48ef-937c-653789e97e4c`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:11:47,068::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:48,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:48,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120532 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:50,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:50,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143144 s, 71.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:50,870::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) (190743) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) (190743) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:50,871::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190743) Building response >Thread-14::DEBUG::2014-08-22 09:11:50,872::__init__::225::IOProcess::(_processLogs) (190743) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:50,872::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:50,872::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:11:50,872::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:11:50,900::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000272765 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:50,901::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:50,901::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:11:50,901::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:50,901::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:50,901::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) (190744) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) (190744) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) (190744) Building response >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) (190744) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:50,902::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:11:50,903::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:50,903::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:11:50,903::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:50,903::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:50,903::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:50,903::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) (190745) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) (190745) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) (190745) Building response >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) (190745) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:50,904::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) (190746) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) (190746) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) (190746) Building response >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) (190746) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:50,905::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:50,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) (190747) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) (190747) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) (190747) Building response >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) (190747) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:50,907::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:51,695::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:51,696::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:51,696::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:11:52,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:52,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123063 s, 83.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:11:54,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:54,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119409 s, 85.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:11:55,292::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:11:55,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`23f6461d-7313-4c48-97ce-43c0c527ed2c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:11:55,293::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:11:55,293::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:55,293::task::1191::Storage.TaskManager.Task::(prepare) Task=`23f6461d-7313-4c48-97ce-43c0c527ed2c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:11:55,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`23f6461d-7313-4c48-97ce-43c0c527ed2c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:11:55,293::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:11:55,293::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:11:55,294::task::993::Storage.TaskManager.Task::(_decref) Task=`23f6461d-7313-4c48-97ce-43c0c527ed2c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:11:55,345::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:11:55,345::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e0a12e7-67ee-4109-af3c-ccbdce6ca44f`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:11:55,345::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:11:55,346::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d0e04af8-5fc5-4091-bd0a-db33aafae0cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:11:55,346::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:11:55,346::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:11:55,346::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d0e04af8-5fc5-4091-bd0a-db33aafae0cf`::Granted request >Thread-171::DEBUG::2014-08-22 09:11:55,346::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2e0a12e7-67ee-4109-af3c-ccbdce6ca44f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:11:55,346::task::993::Storage.TaskManager.Task::(_decref) Task=`2e0a12e7-67ee-4109-af3c-ccbdce6ca44f`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:11:55,348::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:55,348::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e0a12e7-67ee-4109-af3c-ccbdce6ca44f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:11:55,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e0a12e7-67ee-4109-af3c-ccbdce6ca44f`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:11:55,349::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:11:55,349::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:11:55,349::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:11:55,349::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:11:55,349::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:11:55,349::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:11:55,349::task::993::Storage.TaskManager.Task::(_decref) Task=`2e0a12e7-67ee-4109-af3c-ccbdce6ca44f`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:11:56,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:56,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121105 s, 84.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:56,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:56,703::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:11:56,703::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:56,808::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:56,809::__init__::225::IOProcess::(_processLogs) (190748) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:56,809::__init__::225::IOProcess::(_processLogs) (190748) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:56,809::__init__::225::IOProcess::(_processLogs) (190748) Building response >Thread-14::DEBUG::2014-08-22 09:11:56,809::__init__::225::IOProcess::(_processLogs) (190748) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:56,809::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:56,809::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:11:56,809::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:11:56,837::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000251483 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:11:56,838::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:11:56,838::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:11:56,838::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) (190749) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) (190749) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) (190749) Building response >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) (190749) Queuing response >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:11:56,839::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:11:58,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:11:58,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118516 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:00,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:00,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117378 s, 87.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:00,908::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:00,908::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:00,908::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) (190750) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) (190750) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) (190750) Building response >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) (190750) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:00,909::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:12:00,910::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:12:00,938::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0002559 s, 2.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:00,939::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:00,939::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:00,939::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:00,939::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) (190751) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) (190751) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) (190751) Building response >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) (190751) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:00,940::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:12:00,941::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:00,941::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:12:00,941::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:00,941::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:00,941::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) (190752) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) (190752) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) (190752) Building response >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) (190752) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:00,942::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) (190753) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) (190753) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:00,943::__init__::225::IOProcess::(_processLogs) (190753) Building response >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) (190753) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:00,944::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) (190754) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) (190754) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) (190754) Building response >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) (190754) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:00,945::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:01,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:01,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:01,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:12:02,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad27c274-8ad8-47b9-af8f-43a4d46b57c7`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:02,243::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:12:02,243::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000251483', 'lastCheck': '5.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002559', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:02,243::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad27c274-8ad8-47b9-af8f-43a4d46b57c7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000251483', 'lastCheck': '5.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002559', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:02,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad27c274-8ad8-47b9-af8f-43a4d46b57c7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:02,243::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:02,243::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:02,244::task::993::Storage.TaskManager.Task::(_decref) Task=`ad27c274-8ad8-47b9-af8f-43a4d46b57c7`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:12:02,268::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:02,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:02,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119966 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:04,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:04,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129365 s, 79.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:05,405::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:12:05,405::task::595::Storage.TaskManager.Task::(_updateState) Task=`476765d1-f809-4cd6-8b40-7cf04f4586af`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:05,405::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:12:05,405::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:05,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`476765d1-f809-4cd6-8b40-7cf04f4586af`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:05,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`476765d1-f809-4cd6-8b40-7cf04f4586af`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:05,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:05,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:05,406::task::993::Storage.TaskManager.Task::(_decref) Task=`476765d1-f809-4cd6-8b40-7cf04f4586af`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:12:05,439::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:12:05,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ad56d0e-396e-4d68-8e05-c96871020bd6`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:12:05,440::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:12:05,440::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5c55fa82-76f6-42b5-8b99-a75a30846267`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:12:05,440::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:12:05,440::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:12:05,440::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5c55fa82-76f6-42b5-8b99-a75a30846267`::Granted request >Thread-171::DEBUG::2014-08-22 09:12:05,440::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4ad56d0e-396e-4d68-8e05-c96871020bd6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:12:05,441::task::993::Storage.TaskManager.Task::(_decref) Task=`4ad56d0e-396e-4d68-8e05-c96871020bd6`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:12:05,443::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:05,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ad56d0e-396e-4d68-8e05-c96871020bd6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:05,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ad56d0e-396e-4d68-8e05-c96871020bd6`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:12:05,443::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:12:05,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:12:05,443::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:12:05,443::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:12:05,443::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:12:05,443::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:12:05,443::task::993::Storage.TaskManager.Task::(_decref) Task=`4ad56d0e-396e-4d68-8e05-c96871020bd6`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:06,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:06,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:06,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:12:06,840::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) (190755) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) (190755) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) (190755) Building response >Thread-14::DEBUG::2014-08-22 09:12:06,841::__init__::225::IOProcess::(_processLogs) (190755) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:06,842::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:06,842::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:12:06,842::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:12:06,870::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000284155 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:06,871::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:06,871::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:06,871::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:06,871::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:06,871::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) (190756) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) (190756) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) (190756) Building response >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) (190756) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:06,872::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:12:06,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:06,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118727 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:09,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:09,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012139 s, 84.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:10,946::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:10,946::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) (190757) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) (190757) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) (190757) Building response >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) (190757) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:10,947::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:12:10,948::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:12:10,976::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000230016 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:10,977::__init__::225::IOProcess::(_processLogs) (190758) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:10,978::__init__::225::IOProcess::(_processLogs) (190758) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:10,978::__init__::225::IOProcess::(_processLogs) (190758) Building response >Thread-14::DEBUG::2014-08-22 09:12:10,978::__init__::225::IOProcess::(_processLogs) (190758) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:10,978::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:10,978::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) (190759) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:10,979::__init__::225::IOProcess::(_processLogs) (190759) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) (190759) Building response >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) (190759) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:12:10,980::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) (190760) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) (190760) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) (190760) Building response >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) (190760) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:10,981::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) (190761) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:10,982::__init__::225::IOProcess::(_processLogs) (190761) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:10,983::__init__::225::IOProcess::(_processLogs) (190761) Building response >Thread-14::DEBUG::2014-08-22 09:12:10,983::__init__::225::IOProcess::(_processLogs) (190761) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:10,983::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:10,983::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:12:11,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:11,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119041 s, 86.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:11,722::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:11,723::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:11,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:13,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:13,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118976 s, 86.1 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:12:13,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`d15560aa-d9eb-4ab0-b0da-087eefd74586`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:12:13,476::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:12:13,476::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) (190762) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) (190762) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:12:13,477::__init__::225::IOProcess::(_processLogs) DEBUG|(190762) Building response >Thread-14::DEBUG::2014-08-22 09:12:13,478::__init__::225::IOProcess::(_processLogs) (190762) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:13,478::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:13,478::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:12:13,478::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) (190763) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) (190763) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) (190763) Building response >Thread-14::DEBUG::2014-08-22 09:12:13,479::__init__::225::IOProcess::(_processLogs) (190763) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:13,480::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:13,480::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:12:13,480::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:12:13,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`d15560aa-d9eb-4ab0-b0da-087eefd74586`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:12:13,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`d15560aa-d9eb-4ab0-b0da-087eefd74586`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:12:13,480::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:12:13,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:12:13,481::task::993::Storage.TaskManager.Task::(_decref) Task=`d15560aa-d9eb-4ab0-b0da-087eefd74586`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:12:15,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:15,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211816 s, 48.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:15,516::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:12:15,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`6054927b-42fe-44d4-bb74-8641d31c020c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:15,517::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:12:15,517::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:15,517::task::1191::Storage.TaskManager.Task::(prepare) Task=`6054927b-42fe-44d4-bb74-8641d31c020c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:15,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`6054927b-42fe-44d4-bb74-8641d31c020c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:15,518::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:15,518::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:15,518::task::993::Storage.TaskManager.Task::(_decref) Task=`6054927b-42fe-44d4-bb74-8641d31c020c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:12:15,558::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:12:15,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e494f24-de64-4abd-ac6c-ee1b7876c47f`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:12:15,559::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:12:15,559::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`05a0780e-40e0-4f1b-9a89-6ccbb886d65a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:12:15,559::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:12:15,559::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:12:15,559::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`05a0780e-40e0-4f1b-9a89-6ccbb886d65a`::Granted request >Thread-171::DEBUG::2014-08-22 09:12:15,560::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8e494f24-de64-4abd-ac6c-ee1b7876c47f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:12:15,560::task::993::Storage.TaskManager.Task::(_decref) Task=`8e494f24-de64-4abd-ac6c-ee1b7876c47f`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:12:15,562::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:15,562::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e494f24-de64-4abd-ac6c-ee1b7876c47f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:15,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e494f24-de64-4abd-ac6c-ee1b7876c47f`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:12:15,562::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:12:15,562::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:12:15,562::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:12:15,562::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:12:15,562::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:12:15,562::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:12:15,563::task::993::Storage.TaskManager.Task::(_decref) Task=`8e494f24-de64-4abd-ac6c-ee1b7876c47f`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:16,730::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:16,730::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:16,730::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:12:16,873::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:16,873::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) (190764) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) (190764) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) (190764) Building response >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) (190764) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:16,874::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:12:16,875::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:12:16,903::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000228052 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:16,904::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:16,904::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:16,904::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:16,904::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:16,904::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:16,904::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) (190765) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) (190765) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) (190765) Building response >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) (190765) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:16,905::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:12:17,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:17,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117815 s, 86.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:17,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`28ce3977-7b09-4d4a-b8ad-998b778e074e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:17,542::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:12:17,542::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000228052', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000230016', 'lastCheck': '6.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:17,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`28ce3977-7b09-4d4a-b8ad-998b778e074e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000228052', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000230016', 'lastCheck': '6.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:17,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`28ce3977-7b09-4d4a-b8ad-998b778e074e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:17,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:17,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:17,543::task::993::Storage.TaskManager.Task::(_decref) Task=`28ce3977-7b09-4d4a-b8ad-998b778e074e`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:12:17,567::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:19,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:19,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217786 s, 47.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:20,984::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:20,984::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:20,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:20,984::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:20,984::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) (190766) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) (190766) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) (190766) Building response >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) (190766) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:20,985::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:12:20,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:12:21,014::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000268123 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) (190767) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:21,015::__init__::225::IOProcess::(_processLogs) (190767) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:21,016::__init__::225::IOProcess::(_processLogs) (190767) Building response >Thread-14::DEBUG::2014-08-22 09:12:21,016::__init__::225::IOProcess::(_processLogs) (190767) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:21,016::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:21,016::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) (190768) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) (190768) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) (190768) Building response >Thread-14::DEBUG::2014-08-22 09:12:21,017::__init__::225::IOProcess::(_processLogs) (190768) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:21,018::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:21,018::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:21,018::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:21,018::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:12:21,018::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:21,018::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) (190769) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) (190769) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) (190769) Building response >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) (190769) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:21,019::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) (190770) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) (190770) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:21,020::__init__::225::IOProcess::(_processLogs) (190770) Building response >Thread-14::DEBUG::2014-08-22 09:12:21,021::__init__::225::IOProcess::(_processLogs) (190770) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:21,021::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:21,021::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:12:21,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:21,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118287 s, 86.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:21,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:21,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:21,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:23,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:23,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115959 s, 88.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:25,611::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:12:25,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ccc7862-5347-4091-b6eb-81408a7d1baa`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:25,612::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:12:25,612::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:25,612::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ccc7862-5347-4091-b6eb-81408a7d1baa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:25,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ccc7862-5347-4091-b6eb-81408a7d1baa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:25,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:25,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:25,613::task::993::Storage.TaskManager.Task::(_decref) Task=`4ccc7862-5347-4091-b6eb-81408a7d1baa`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:12:25,650::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:12:25,650::task::595::Storage.TaskManager.Task::(_updateState) Task=`be1d6436-2ada-4d3b-a31f-00686f38b174`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:12:25,651::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:12:25,651::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`283f2ebf-268d-4ab8-8459-97f10bad9b52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:12:25,651::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:12:25,651::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:12:25,651::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`283f2ebf-268d-4ab8-8459-97f10bad9b52`::Granted request >Thread-171::DEBUG::2014-08-22 09:12:25,651::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`be1d6436-2ada-4d3b-a31f-00686f38b174`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:12:25,652::task::993::Storage.TaskManager.Task::(_decref) Task=`be1d6436-2ada-4d3b-a31f-00686f38b174`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:12:25,654::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:25,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`be1d6436-2ada-4d3b-a31f-00686f38b174`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:25,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`be1d6436-2ada-4d3b-a31f-00686f38b174`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:12:25,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:12:25,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:12:25,654::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:12:25,654::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:12:25,654::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:12:25,654::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:12:25,654::task::993::Storage.TaskManager.Task::(_decref) Task=`be1d6436-2ada-4d3b-a31f-00686f38b174`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:12:25,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:25,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118006 s, 86.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:26,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:26,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:26,744::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:12:26,906::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:26,906::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:26,906::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) (190771) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) (190771) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) (190771) Building response >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) (190771) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:12:26,907::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:12:26,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:12:26,936::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000265076 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:26,937::__init__::225::IOProcess::(_processLogs) (190772) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:26,938::__init__::225::IOProcess::(_processLogs) (190772) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:26,938::__init__::225::IOProcess::(_processLogs) (190772) Building response >Thread-14::DEBUG::2014-08-22 09:12:26,938::__init__::225::IOProcess::(_processLogs) (190772) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:26,938::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:26,938::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:12:27,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:27,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119654 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:29,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:29,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181104 s, 56.5 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-22 09:12:31,021::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-14::DEBUG::2014-08-22 09:12:31,023::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,023::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-14::DEBUG::2014-08-22 09:12:31,023::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,023::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) (190773) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) (190773) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) (190773) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) (190773) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,024::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-14::DEBUG::2014-08-22 09:12:31,025::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,025::__init__::225::IOProcess::(_processLogs) Message size is 145 >Thread-14::DEBUG::2014-08-22 09:12:31,025::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,025::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,025::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,025::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) (190774) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) (190774) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) DEBUG|(190774) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) (190774) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,026::__init__::225::IOProcess::(_processLogs) Sending response sized 195 >Thread-16::DEBUG::2014-08-22 09:12:31,028::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-14::DEBUG::2014-08-22 09:12:31,028::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,028::__init__::225::IOProcess::(_processLogs) Message size is 174 >Thread-14::DEBUG::2014-08-22 09:12:31,028::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,029::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,029::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,029::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,029::__init__::225::IOProcess::(_processLogs) (190775) Finding callback 'touch'... >Thread-14::DEBUG::2014-08-22 09:12:31,029::__init__::225::IOProcess::(_processLogs) (190775) Got request for method 'touch' >Thread-14::DEBUG::2014-08-22 09:12:31,056::__init__::225::IOProcess::(_processLogs) DEBUG|(190775) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,056::__init__::225::IOProcess::(_processLogs) (190775) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,056::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,056::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-16::DEBUG::2014-08-22 09:12:31,056::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) Message size is 198 >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) (190776) Finding callback 'readfile'... >Thread-14::DEBUG::2014-08-22 09:12:31,057::__init__::225::IOProcess::(_processLogs) (190776) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-22 09:12:31,058::__init__::225::IOProcess::(_processLogs) DEBUG|(190776) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,058::__init__::225::IOProcess::(_processLogs) (190776) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,058::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,058::__init__::225::IOProcess::(_processLogs) Sending response sized 834 >Thread-16::DEBUG::2014-08-22 09:12:31,058::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) Message size is 191 >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) (190777) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) (190777) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) (190777) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,059::__init__::225::IOProcess::(_processLogs) (190777) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,060::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) (190778) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) (190778) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) (190778) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) (190778) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,061::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-16::DEBUG::2014-08-22 09:12:31,061::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-22 09:12:31,061::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-16::INFO::2014-08-22 09:12:31,062::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-14::DEBUG::2014-08-22 09:12:31,062::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,062::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:31,062::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,062::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,062::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,062::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,063::__init__::225::IOProcess::(_processLogs) (190779) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:31,063::__init__::225::IOProcess::(_processLogs) (190779) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:31,063::__init__::225::IOProcess::(_processLogs) (190779) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,063::__init__::225::IOProcess::(_processLogs) (190779) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,063::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,063::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:12:31,063::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:12:31,092::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000307448 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:31,092::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) (190780) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) (190780) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) (190780) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,093::__init__::225::IOProcess::(_processLogs) (190780) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,094::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,094::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:12:31,094::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) (190781) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) (190781) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) (190781) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) (190781) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,095::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,096::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) (190782) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) (190782) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) (190782) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) (190782) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:31,097::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) (190783) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) (190783) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) (190783) Building response >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) (190783) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:31,098::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:31,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:31,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:31,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:31,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:32,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143583 s, 71.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:32,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`47c7ed40-e330-423e-96f6-62b58264c384`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:32,785::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:12:32,785::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000265076', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000307448', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:32,786::task::1191::Storage.TaskManager.Task::(prepare) Task=`47c7ed40-e330-423e-96f6-62b58264c384`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000265076', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000307448', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:32,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`47c7ed40-e330-423e-96f6-62b58264c384`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:32,786::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:32,786::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:32,786::task::993::Storage.TaskManager.Task::(_decref) Task=`47c7ed40-e330-423e-96f6-62b58264c384`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:12:32,809::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:34,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:34,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118095 s, 86.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:35,699::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:12:35,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`440f11fb-3ea2-4668-8a8d-18a35c7252f1`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:35,699::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:12:35,700::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:35,700::task::1191::Storage.TaskManager.Task::(prepare) Task=`440f11fb-3ea2-4668-8a8d-18a35c7252f1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:35,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`440f11fb-3ea2-4668-8a8d-18a35c7252f1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:35,700::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:35,700::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:35,700::task::993::Storage.TaskManager.Task::(_decref) Task=`440f11fb-3ea2-4668-8a8d-18a35c7252f1`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:12:35,737::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:12:35,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`61a4550f-f070-4971-a59a-0b17434b59a4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:12:35,737::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:12:35,737::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4dae7c86-55b0-4bf5-b077-a1522ea4bac9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:12:35,737::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:12:35,738::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:12:35,738::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4dae7c86-55b0-4bf5-b077-a1522ea4bac9`::Granted request >Thread-171::DEBUG::2014-08-22 09:12:35,738::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`61a4550f-f070-4971-a59a-0b17434b59a4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:12:35,738::task::993::Storage.TaskManager.Task::(_decref) Task=`61a4550f-f070-4971-a59a-0b17434b59a4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:12:35,740::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:35,740::task::1191::Storage.TaskManager.Task::(prepare) Task=`61a4550f-f070-4971-a59a-0b17434b59a4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:35,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`61a4550f-f070-4971-a59a-0b17434b59a4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:12:35,740::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:12:35,740::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:12:35,741::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:12:35,741::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:12:35,741::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:12:35,741::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:12:35,741::task::993::Storage.TaskManager.Task::(_decref) Task=`61a4550f-f070-4971-a59a-0b17434b59a4`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:12:36,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:36,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115303 s, 88.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:36,757::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:36,757::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:36,758::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:12:36,939::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:36,939::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:36,939::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:36,939::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:36,939::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) (190784) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) (190784) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) DEBUG|(190784) Building response >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) (190784) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:36,940::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:12:36,941::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:12:36,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199832 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) (190785) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:36,970::__init__::225::IOProcess::(_processLogs) (190785) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:36,971::__init__::225::IOProcess::(_processLogs) (190785) Building response >Thread-14::DEBUG::2014-08-22 09:12:36,971::__init__::225::IOProcess::(_processLogs) (190785) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:36,971::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:36,971::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:12:38,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:38,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121711 s, 84.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:40,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:40,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115513 s, 88.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) (190786) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) (190786) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:41,100::__init__::225::IOProcess::(_processLogs) (190786) Building response >Thread-14::DEBUG::2014-08-22 09:12:41,101::__init__::225::IOProcess::(_processLogs) (190786) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:41,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:41,101::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:12:41,101::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:12:41,129::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00025294 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:41,130::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:41,130::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:41,130::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:41,130::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) (190787) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) (190787) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) (190787) Building response >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) (190787) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:41,131::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:12:41,132::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:41,132::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:12:41,132::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:41,132::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:41,132::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) (190788) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) (190788) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) (190788) Building response >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) (190788) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:41,133::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) (190789) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) (190789) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:41,134::__init__::225::IOProcess::(_processLogs) (190789) Building response >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) (190789) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:41,135::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) (190790) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) (190790) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) (190790) Building response >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) (190790) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:41,136::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:41,765::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:41,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:41,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:42,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:42,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120737 s, 84.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:44,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:44,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117428 s, 87.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:45,806::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:12:45,806::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdacb84a-4611-4573-94af-d8b80771453c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:45,806::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:12:45,806::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:45,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdacb84a-4611-4573-94af-d8b80771453c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:45,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdacb84a-4611-4573-94af-d8b80771453c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:45,807::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:45,807::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:45,807::task::993::Storage.TaskManager.Task::(_decref) Task=`bdacb84a-4611-4573-94af-d8b80771453c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:12:45,848::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:12:45,849::task::595::Storage.TaskManager.Task::(_updateState) Task=`536b282d-1731-4ded-a229-67edb21e1e06`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:12:45,849::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:12:45,849::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`de59d1bd-9b4a-4dd1-952b-9c3e64e60e31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:12:45,849::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:12:45,849::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:12:45,849::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`de59d1bd-9b4a-4dd1-952b-9c3e64e60e31`::Granted request >Thread-171::DEBUG::2014-08-22 09:12:45,850::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`536b282d-1731-4ded-a229-67edb21e1e06`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:12:45,850::task::993::Storage.TaskManager.Task::(_decref) Task=`536b282d-1731-4ded-a229-67edb21e1e06`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:12:45,852::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:45,852::task::1191::Storage.TaskManager.Task::(prepare) Task=`536b282d-1731-4ded-a229-67edb21e1e06`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:45,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`536b282d-1731-4ded-a229-67edb21e1e06`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:12:45,852::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:12:45,852::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:12:45,852::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:12:45,852::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:12:45,853::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:12:45,853::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:12:45,853::task::993::Storage.TaskManager.Task::(_decref) Task=`536b282d-1731-4ded-a229-67edb21e1e06`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:12:46,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:46,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119999 s, 85.3 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:46,772::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:46,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:46,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:12:46,972::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:46,972::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:46,972::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:46,972::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:46,972::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) (190791) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) (190791) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) (190791) Building response >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) (190791) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:12:46,973::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:12:46,974::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:12:47,002::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227866 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:47,002::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) (190792) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) (190792) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) (190792) Building response >Thread-14::DEBUG::2014-08-22 09:12:47,003::__init__::225::IOProcess::(_processLogs) (190792) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:47,004::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:47,004::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:12:47,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`e322ca37-b8c8-4090-a5f3-fa7f1ca6f8fd`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:47,975::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:12:47,977::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227866', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00025294', 'lastCheck': '6.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:47,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`e322ca37-b8c8-4090-a5f3-fa7f1ca6f8fd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227866', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00025294', 'lastCheck': '6.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:12:47,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`e322ca37-b8c8-4090-a5f3-fa7f1ca6f8fd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:47,978::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:47,978::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:47,979::task::993::Storage.TaskManager.Task::(_decref) Task=`e322ca37-b8c8-4090-a5f3-fa7f1ca6f8fd`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:12:48,005::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:48,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:48,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120366 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:50,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:50,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116793 s, 87.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:51,137::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:51,137::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) (190793) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) (190793) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) (190793) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:51,138::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:12:51,139::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:12:51,168::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000168689 s, 3.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:51,170::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) (190794) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) (190794) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) (190794) Building response >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) (190794) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:51,171::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:51,172::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:12:51,172::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:12:51,172::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) (190795) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) (190795) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) (190795) Building response >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) (190795) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:51,173::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) (190796) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) (190796) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:51,174::__init__::225::IOProcess::(_processLogs) (190796) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:51,175::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:51,175::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:12:51,175::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:12:51,175::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:51,175::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) (190797) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) (190797) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) (190797) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:51,176::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:51,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:51,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:51,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:52,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:52,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120421 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:12:54,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:54,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116912 s, 87.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:12:55,901::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:12:55,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ca7557a-57d6-4643-a4d9-55537435707e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:12:55,902::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:12:55,902::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:55,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ca7557a-57d6-4643-a4d9-55537435707e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:12:55,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ca7557a-57d6-4643-a4d9-55537435707e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:12:55,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:12:55,902::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:12:55,902::task::993::Storage.TaskManager.Task::(_decref) Task=`5ca7557a-57d6-4643-a4d9-55537435707e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:12:55,936::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:12:55,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`29493bd8-7598-4811-890e-a73e4b9e6172`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:12:55,937::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:12:55,937::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f10c8324-c775-45fd-a0fe-86c1e06e9918`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:12:55,937::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:12:55,937::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:12:55,937::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f10c8324-c775-45fd-a0fe-86c1e06e9918`::Granted request >Thread-171::DEBUG::2014-08-22 09:12:55,937::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`29493bd8-7598-4811-890e-a73e4b9e6172`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:12:55,938::task::993::Storage.TaskManager.Task::(_decref) Task=`29493bd8-7598-4811-890e-a73e4b9e6172`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:12:55,940::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:55,940::task::1191::Storage.TaskManager.Task::(prepare) Task=`29493bd8-7598-4811-890e-a73e4b9e6172`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:12:55,940::task::595::Storage.TaskManager.Task::(_updateState) Task=`29493bd8-7598-4811-890e-a73e4b9e6172`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:12:55,940::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:12:55,940::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:12:55,940::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:12:55,940::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:12:55,940::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:12:55,940::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:12:55,941::task::993::Storage.TaskManager.Task::(_decref) Task=`29493bd8-7598-4811-890e-a73e4b9e6172`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:56,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:56,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:12:56,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:12:56,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:56,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117322 s, 87.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) (190798) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:57,005::__init__::225::IOProcess::(_processLogs) (190798) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:57,006::__init__::225::IOProcess::(_processLogs) (190798) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:57,006::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:57,006::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:12:57,006::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:12:57,034::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000193261 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:12:57,035::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:12:57,036::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:12:57,036::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:12:57,036::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:12:57,036::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:12:57,036::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:12:57,036::__init__::225::IOProcess::(_processLogs) (190799) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:12:57,037::__init__::225::IOProcess::(_processLogs) (190799) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:12:57,037::__init__::225::IOProcess::(_processLogs) (190799) Building response >Thread-14::DEBUG::2014-08-22 09:12:57,037::__init__::225::IOProcess::(_processLogs) (190799) Queuing response >Thread-14::DEBUG::2014-08-22 09:12:57,037::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:12:57,037::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:12:58,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:12:59,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121241 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:01,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:01,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118412 s, 86.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:01,177::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:01,177::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:01,177::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) (190800) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) (190800) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) (190800) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:01,178::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:13:01,179::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:13:01,208::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000238569 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) (190801) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) (190801) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) (190801) Building response >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) (190801) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:01,209::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:01,210::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:13:01,210::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:01,210::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) (190802) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) (190802) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) (190802) Building response >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) (190802) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:01,211::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) (190803) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) (190803) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) (190803) Building response >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) (190803) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:01,212::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:01,213::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:01,213::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:13:01,213::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:01,213::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:01,213::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:01,213::__init__::225::IOProcess::(_processLogs) (190804) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:01,214::__init__::225::IOProcess::(_processLogs) (190804) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:01,214::__init__::225::IOProcess::(_processLogs) (190804) Building response >Thread-14::DEBUG::2014-08-22 09:13:01,214::__init__::225::IOProcess::(_processLogs) (190804) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:01,214::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:01,214::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:01,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:01,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:01,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:03,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:03,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118327 s, 86.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:03,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b2c7252-1f0b-4e0f-8532-7b789d0bd2dd`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:03,222::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:13:03,222::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193261', 'lastCheck': '6.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000238569', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:03,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b2c7252-1f0b-4e0f-8532-7b789d0bd2dd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193261', 'lastCheck': '6.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000238569', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:03,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b2c7252-1f0b-4e0f-8532-7b789d0bd2dd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:03,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:03,222::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:03,222::task::993::Storage.TaskManager.Task::(_decref) Task=`7b2c7252-1f0b-4e0f-8532-7b789d0bd2dd`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:13:03,247::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:05,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:05,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116468 s, 87.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:06,006::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-13::DEBUG::2014-08-22 09:13:06,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbbe3797-398b-40db-a33d-21551dc55a52`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:06,007::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:13:06,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:06,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`dbbe3797-398b-40db-a33d-21551dc55a52`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:06,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbbe3797-398b-40db-a33d-21551dc55a52`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:06,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:06,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:06,008::task::993::Storage.TaskManager.Task::(_decref) Task=`dbbe3797-398b-40db-a33d-21551dc55a52`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:13:06,125::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-171::DEBUG::2014-08-22 09:13:06,125::task::595::Storage.TaskManager.Task::(_updateState) Task=`54c8eea9-bc55-4071-bfb8-e57886eddf4b`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:13:06,125::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:13:06,125::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`77e2136d-0cfc-4bd4-91e7-5a39c2681e62`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:13:06,125::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:13:06,126::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:13:06,126::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`77e2136d-0cfc-4bd4-91e7-5a39c2681e62`::Granted request >Thread-171::DEBUG::2014-08-22 09:13:06,126::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`54c8eea9-bc55-4071-bfb8-e57886eddf4b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:13:06,126::task::993::Storage.TaskManager.Task::(_decref) Task=`54c8eea9-bc55-4071-bfb8-e57886eddf4b`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:13:06,128::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:06,128::task::1191::Storage.TaskManager.Task::(prepare) Task=`54c8eea9-bc55-4071-bfb8-e57886eddf4b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:06,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`54c8eea9-bc55-4071-bfb8-e57886eddf4b`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:13:06,128::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:13:06,128::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:13:06,129::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:13:06,129::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:13:06,129::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:13:06,129::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:13:06,129::task::993::Storage.TaskManager.Task::(_decref) Task=`54c8eea9-bc55-4071-bfb8-e57886eddf4b`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:06,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:06,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:06,800::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:13:07,038::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:07,038::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) (190805) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) (190805) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) (190805) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:07,039::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:13:07,040::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:13:07,068::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00027437 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) (190806) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) (190806) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) (190806) Building response >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) (190806) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:07,069::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:13:07,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:07,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117586 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:09,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:09,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118187 s, 86.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:11,215::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:11,215::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:11,215::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) (190807) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) (190807) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) (190807) Building response >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) (190807) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:11,216::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:13:11,217::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:13:11,245::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000205886 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) (190808) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) (190808) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) (190808) Building response >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) (190808) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:11,246::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:13:11,247::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:13:11,247::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) (190809) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) (190809) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) (190809) Building response >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) (190809) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:11,248::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) (190810) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) (190810) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) (190810) Building response >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) (190810) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:11,249::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:11,250::__init__::225::IOProcess::(_processLogs) (190811) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:11,251::__init__::225::IOProcess::(_processLogs) (190811) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:11,251::__init__::225::IOProcess::(_processLogs) (190811) Building response >Thread-14::DEBUG::2014-08-22 09:13:11,251::__init__::225::IOProcess::(_processLogs) (190811) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:11,251::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:11,251::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:13:11,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:11,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118581 s, 86.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:11,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:11,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:11,807::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:13,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:13,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011737 s, 87.2 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:13:13,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f24221b-b602-481d-aeab-9c510f8b90b8`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:13:13,531::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) (190812) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:13:13,532::__init__::225::IOProcess::(_processLogs) (190812) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:13:13,533::__init__::225::IOProcess::(_processLogs) DEBUG|(190812) Building response >Thread-14::DEBUG::2014-08-22 09:13:13,533::__init__::225::IOProcess::(_processLogs) (190812) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:13,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:13,533::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) (190813) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) (190813) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:13:13,534::__init__::225::IOProcess::(_processLogs) (190813) Building response >Thread-14::DEBUG::2014-08-22 09:13:13,535::__init__::225::IOProcess::(_processLogs) (190813) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:13,535::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:13,535::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:13:13,535::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:13:13,535::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f24221b-b602-481d-aeab-9c510f8b90b8`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:13:13,535::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f24221b-b602-481d-aeab-9c510f8b90b8`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:13:13,535::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:13:13,535::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:13:13,536::task::993::Storage.TaskManager.Task::(_decref) Task=`1f24221b-b602-481d-aeab-9c510f8b90b8`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:13:15,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:15,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118178 s, 86.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:16,185::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:13:16,186::task::595::Storage.TaskManager.Task::(_updateState) Task=`245b5b18-feff-4b5e-a2f8-9e3b407b6c4a`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:16,186::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:13:16,186::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:16,186::task::1191::Storage.TaskManager.Task::(prepare) Task=`245b5b18-feff-4b5e-a2f8-9e3b407b6c4a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:16,186::task::595::Storage.TaskManager.Task::(_updateState) Task=`245b5b18-feff-4b5e-a2f8-9e3b407b6c4a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:16,187::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:16,187::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:16,187::task::993::Storage.TaskManager.Task::(_decref) Task=`245b5b18-feff-4b5e-a2f8-9e3b407b6c4a`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:13:16,222::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:13:16,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`424dc866-6bdb-40f2-b50f-7f07476a1db4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:13:16,223::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:13:16,223::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`68345da4-f7e5-4c7a-b047-7b4a4c73e2d7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:13:16,223::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:13:16,223::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:13:16,223::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`68345da4-f7e5-4c7a-b047-7b4a4c73e2d7`::Granted request >Thread-171::DEBUG::2014-08-22 09:13:16,224::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`424dc866-6bdb-40f2-b50f-7f07476a1db4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:13:16,224::task::993::Storage.TaskManager.Task::(_decref) Task=`424dc866-6bdb-40f2-b50f-7f07476a1db4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:13:16,226::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:16,226::task::1191::Storage.TaskManager.Task::(prepare) Task=`424dc866-6bdb-40f2-b50f-7f07476a1db4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:16,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`424dc866-6bdb-40f2-b50f-7f07476a1db4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:13:16,226::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:13:16,226::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:13:16,226::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:13:16,226::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:13:16,227::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:13:16,227::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:13:16,227::task::993::Storage.TaskManager.Task::(_decref) Task=`424dc866-6bdb-40f2-b50f-7f07476a1db4`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:16,812::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:16,813::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:16,813::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) (190814) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) (190814) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:17,071::__init__::225::IOProcess::(_processLogs) DEBUG|(190814) Building response >Thread-14::DEBUG::2014-08-22 09:13:17,072::__init__::225::IOProcess::(_processLogs) (190814) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:17,072::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:17,072::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:13:17,072::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:13:17,100::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000190956 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:17,101::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:17,101::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:17,101::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:17,101::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:17,101::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:17,101::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) (190815) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) (190815) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) (190815) Building response >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) (190815) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:17,102::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:13:17,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:17,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121803 s, 84.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:18,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`230ffd18-38d4-4d12-a359-c8116a1444ba`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:18,470::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:13:18,470::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000190956', 'lastCheck': '1.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000205886', 'lastCheck': '7.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:18,470::task::1191::Storage.TaskManager.Task::(prepare) Task=`230ffd18-38d4-4d12-a359-c8116a1444ba`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000190956', 'lastCheck': '1.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000205886', 'lastCheck': '7.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:18,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`230ffd18-38d4-4d12-a359-c8116a1444ba`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:18,470::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:18,470::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:18,470::task::993::Storage.TaskManager.Task::(_decref) Task=`230ffd18-38d4-4d12-a359-c8116a1444ba`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:13:18,496::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:19,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:19,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214138 s, 47.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:21,252::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:21,252::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:21,252::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:21,252::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) (190816) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) (190816) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) (190816) Building response >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) (190816) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:21,253::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:13:21,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:13:21,282::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000296752 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) (190817) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) (190817) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:21,283::__init__::225::IOProcess::(_processLogs) (190817) Building response >Thread-14::DEBUG::2014-08-22 09:13:21,284::__init__::225::IOProcess::(_processLogs) (190817) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:21,284::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:21,284::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) (190818) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) (190818) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) (190818) Building response >Thread-14::DEBUG::2014-08-22 09:13:21,285::__init__::225::IOProcess::(_processLogs) (190818) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:21,286::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) (190819) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) (190819) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) (190819) Building response >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) (190819) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:21,287::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) (190820) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) (190820) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:21,288::__init__::225::IOProcess::(_processLogs) (190820) Building response >Thread-14::DEBUG::2014-08-22 09:13:21,289::__init__::225::IOProcess::(_processLogs) (190820) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:21,289::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:21,289::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:21,819::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:21,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:21,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:21,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:21,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117652 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:23,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:24,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116096 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:26,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:26,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117602 s, 87.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:26,272::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:13:26,273::task::595::Storage.TaskManager.Task::(_updateState) Task=`2864fd95-4b35-4741-bed4-b0842908945c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:26,273::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:13:26,273::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:26,273::task::1191::Storage.TaskManager.Task::(prepare) Task=`2864fd95-4b35-4741-bed4-b0842908945c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:26,273::task::595::Storage.TaskManager.Task::(_updateState) Task=`2864fd95-4b35-4741-bed4-b0842908945c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:26,273::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:26,274::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:26,274::task::993::Storage.TaskManager.Task::(_decref) Task=`2864fd95-4b35-4741-bed4-b0842908945c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:13:26,312::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:13:26,313::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9dfc690-bb50-4e77-915d-227d0a2814e5`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:13:26,313::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:13:26,313::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`50d47af0-f933-404f-9182-a18ede76bf04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:13:26,313::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:13:26,313::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:13:26,313::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`50d47af0-f933-404f-9182-a18ede76bf04`::Granted request >Thread-171::DEBUG::2014-08-22 09:13:26,314::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b9dfc690-bb50-4e77-915d-227d0a2814e5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:13:26,314::task::993::Storage.TaskManager.Task::(_decref) Task=`b9dfc690-bb50-4e77-915d-227d0a2814e5`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:13:26,316::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:26,316::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9dfc690-bb50-4e77-915d-227d0a2814e5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:26,316::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9dfc690-bb50-4e77-915d-227d0a2814e5`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:13:26,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:13:26,316::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:13:26,316::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:13:26,316::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:13:26,316::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:13:26,316::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:13:26,317::task::993::Storage.TaskManager.Task::(_decref) Task=`b9dfc690-bb50-4e77-915d-227d0a2814e5`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:26,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:26,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:26,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:13:27,103::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:27,103::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:27,103::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) (190821) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) (190821) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) (190821) Building response >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) (190821) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:13:27,104::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:13:27,105::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:13:27,133::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000234455 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) (190822) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) (190822) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) (190822) Building response >Thread-14::DEBUG::2014-08-22 09:13:27,134::__init__::225::IOProcess::(_processLogs) (190822) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:27,135::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:27,135::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:13:28,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:28,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119733 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:30,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:30,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011617 s, 88.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:31,290::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:31,290::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:31,290::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:31,290::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:31,290::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:31,290::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) (190823) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) (190823) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) (190823) Building response >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) (190823) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:31,291::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:13:31,292::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:13:31,320::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000312261 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) (190824) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:31,321::__init__::225::IOProcess::(_processLogs) (190824) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:31,322::__init__::225::IOProcess::(_processLogs) (190824) Building response >Thread-14::DEBUG::2014-08-22 09:13:31,322::__init__::225::IOProcess::(_processLogs) (190824) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:31,322::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:31,322::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) (190825) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) (190825) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:31,323::__init__::225::IOProcess::(_processLogs) (190825) Building response >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) (190825) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:31,324::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) (190826) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) (190826) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) (190826) Building response >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) (190826) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:31,325::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) (190827) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) (190827) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:31,326::__init__::225::IOProcess::(_processLogs) (190827) Building response >Thread-14::DEBUG::2014-08-22 09:13:31,327::__init__::225::IOProcess::(_processLogs) (190827) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:31,327::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:31,327::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:31,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:31,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:31,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:32,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:32,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01189 s, 86.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:33,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c9a6aa2-6f2d-446b-baf7-049207123b3f`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:33,676::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:13:33,676::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000234455', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000312261', 'lastCheck': '2.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:33,676::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c9a6aa2-6f2d-446b-baf7-049207123b3f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000234455', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000312261', 'lastCheck': '2.3', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:33,676::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c9a6aa2-6f2d-446b-baf7-049207123b3f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:33,676::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:33,676::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:33,676::task::993::Storage.TaskManager.Task::(_decref) Task=`4c9a6aa2-6f2d-446b-baf7-049207123b3f`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:13:33,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:34,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:34,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124776 s, 82.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:36,381::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:13:36,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`001939ba-6c82-40b2-88f5-da6f74481513`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:36,381::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:13:36,382::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:36,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`001939ba-6c82-40b2-88f5-da6f74481513`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:36,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`001939ba-6c82-40b2-88f5-da6f74481513`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:36,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:36,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:36,382::task::993::Storage.TaskManager.Task::(_decref) Task=`001939ba-6c82-40b2-88f5-da6f74481513`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:13:36,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-171::DEBUG::2014-08-22 09:13:36,416::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:13:36,416::task::595::Storage.TaskManager.Task::(_updateState) Task=`27eeada1-5f31-4303-9ddc-26c86c827784`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:13:36,416::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:13:36,417::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b756be63-bd65-40a1-9519-7636cf0fb7da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:13:36,417::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:13:36,417::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:13:36,417::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b756be63-bd65-40a1-9519-7636cf0fb7da`::Granted request >Thread-171::DEBUG::2014-08-22 09:13:36,418::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`27eeada1-5f31-4303-9ddc-26c86c827784`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:13:36,418::task::993::Storage.TaskManager.Task::(_decref) Task=`27eeada1-5f31-4303-9ddc-26c86c827784`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:13:36,420::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:36,420::task::1191::Storage.TaskManager.Task::(prepare) Task=`27eeada1-5f31-4303-9ddc-26c86c827784`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:36,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`27eeada1-5f31-4303-9ddc-26c86c827784`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:13:36,420::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:13:36,420::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:13:36,420::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:13:36,420::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:13:36,420::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:13:36,421::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:13:36,421::task::993::Storage.TaskManager.Task::(_decref) Task=`27eeada1-5f31-4303-9ddc-26c86c827784`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:13:36,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117854 s, 86.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:36,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:36,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:36,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:37,136::__init__::225::IOProcess::(_processLogs) (190828) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:37,137::__init__::225::IOProcess::(_processLogs) (190828) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:37,137::__init__::225::IOProcess::(_processLogs) DEBUG|(190828) Building response >Thread-14::DEBUG::2014-08-22 09:13:37,137::__init__::225::IOProcess::(_processLogs) (190828) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:37,137::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:37,137::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:13:37,137::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:13:37,165::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000280277 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:37,166::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:37,166::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:37,166::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:37,166::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) (190829) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) (190829) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) (190829) Building response >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) (190829) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:37,167::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:13:38,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:38,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120386 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:40,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:40,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116561 s, 87.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:41,328::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:41,328::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:41,328::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:41,328::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:41,328::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) (190830) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) (190830) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) (190830) Building response >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) (190830) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:41,329::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:13:41,330::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:13:41,358::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000209255 s, 2.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:41,358::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:41,358::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:41,358::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) (190831) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) (190831) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) (190831) Building response >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) (190831) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:41,359::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:13:41,360::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:41,360::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:13:41,360::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:41,360::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) (190832) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) (190832) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) (190832) Building response >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) (190832) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:41,361::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) (190833) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:41,362::__init__::225::IOProcess::(_processLogs) (190833) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) (190833) Building response >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) (190833) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:13:41,363::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) (190834) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) (190834) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) (190834) Building response >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) (190834) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:41,364::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:41,846::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:41,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:41,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:42,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:42,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120231 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:44,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:44,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118198 s, 86.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:46,464::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:13:46,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`5de0b366-d61f-45b4-8955-35f2dfe5e274`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:46,465::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:13:46,465::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:46,465::task::1191::Storage.TaskManager.Task::(prepare) Task=`5de0b366-d61f-45b4-8955-35f2dfe5e274`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:46,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`5de0b366-d61f-45b4-8955-35f2dfe5e274`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:46,465::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:46,466::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:46,466::task::993::Storage.TaskManager.Task::(_decref) Task=`5de0b366-d61f-45b4-8955-35f2dfe5e274`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:13:46,501::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:13:46,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`f27496cc-6c84-46ed-86dc-7e9bbbf83eaa`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:13:46,501::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:13:46,502::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`65bfb877-54d9-42a3-881a-a3607e16311f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:13:46,502::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:13:46,502::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:13:46,502::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`65bfb877-54d9-42a3-881a-a3607e16311f`::Granted request >Thread-171::DEBUG::2014-08-22 09:13:46,502::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f27496cc-6c84-46ed-86dc-7e9bbbf83eaa`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:13:46,502::task::993::Storage.TaskManager.Task::(_decref) Task=`f27496cc-6c84-46ed-86dc-7e9bbbf83eaa`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:13:46,504::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:46,505::task::1191::Storage.TaskManager.Task::(prepare) Task=`f27496cc-6c84-46ed-86dc-7e9bbbf83eaa`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:46,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`f27496cc-6c84-46ed-86dc-7e9bbbf83eaa`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:13:46,505::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:13:46,505::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:13:46,505::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:13:46,505::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:13:46,505::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:13:46,505::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:13:46,505::task::993::Storage.TaskManager.Task::(_decref) Task=`f27496cc-6c84-46ed-86dc-7e9bbbf83eaa`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:13:46,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:46,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116407 s, 88.0 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:46,853::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:46,853::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:46,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:13:47,168::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) (190835) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) (190835) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) DEBUG|(190835) Building response >Thread-14::DEBUG::2014-08-22 09:13:47,169::__init__::225::IOProcess::(_processLogs) (190835) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:47,170::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:47,170::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:13:47,170::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:13:47,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000236321 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:47,199::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:47,200::__init__::225::IOProcess::(_processLogs) (190836) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:47,200::__init__::225::IOProcess::(_processLogs) (190836) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:47,200::__init__::225::IOProcess::(_processLogs) (190836) Building response >Thread-14::DEBUG::2014-08-22 09:13:47,200::__init__::225::IOProcess::(_processLogs) (190836) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:47,200::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:47,200::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:13:48,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:48,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120376 s, 85.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:48,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`9388b490-6479-4cd8-bef4-9cfdbb6abb34`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:48,899::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:13:48,900::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000236321', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209255', 'lastCheck': '7.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:48,901::task::1191::Storage.TaskManager.Task::(prepare) Task=`9388b490-6479-4cd8-bef4-9cfdbb6abb34`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000236321', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209255', 'lastCheck': '7.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:13:48,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`9388b490-6479-4cd8-bef4-9cfdbb6abb34`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:48,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:48,902::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:48,903::task::993::Storage.TaskManager.Task::(_decref) Task=`9388b490-6479-4cd8-bef4-9cfdbb6abb34`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:13:48,939::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:50,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:50,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130038 s, 78.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) (190837) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:51,366::__init__::225::IOProcess::(_processLogs) (190837) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:51,367::__init__::225::IOProcess::(_processLogs) (190837) Building response >Thread-14::DEBUG::2014-08-22 09:13:51,367::__init__::225::IOProcess::(_processLogs) (190837) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:51,367::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:51,367::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:13:51,367::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:13:51,396::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000224618 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:51,396::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:51,396::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:13:51,396::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) (190838) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) (190838) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) (190838) Building response >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) (190838) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:51,397::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:13:51,398::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:51,398::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:13:51,398::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:51,398::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) (190839) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) (190839) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) (190839) Building response >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) (190839) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:51,399::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) (190840) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) (190840) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:51,400::__init__::225::IOProcess::(_processLogs) (190840) Building response >Thread-14::DEBUG::2014-08-22 09:13:51,401::__init__::225::IOProcess::(_processLogs) (190840) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:51,401::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:51,401::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:13:51,401::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:51,401::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:13:51,401::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) (190841) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) (190841) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) (190841) Building response >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) (190841) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:51,402::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:51,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:51,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:51,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:53,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:53,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117945 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:13:55,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:55,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114866 s, 89.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:13:56,568::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-13::DEBUG::2014-08-22 09:13:56,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c0b524f-2eef-48c9-b952-e2068dde8e6c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:13:56,568::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:13:56,569::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:56,569::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c0b524f-2eef-48c9-b952-e2068dde8e6c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:13:56,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c0b524f-2eef-48c9-b952-e2068dde8e6c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:13:56,569::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:13:56,569::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:13:56,569::task::993::Storage.TaskManager.Task::(_decref) Task=`2c0b524f-2eef-48c9-b952-e2068dde8e6c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:13:56,601::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-171::DEBUG::2014-08-22 09:13:56,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e828223-4b11-430f-bd68-c84ff812f9a5`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:13:56,602::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:13:56,602::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1a266dc7-5319-4ae6-85fc-df8030222131`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:13:56,602::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:13:56,602::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:13:56,602::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1a266dc7-5319-4ae6-85fc-df8030222131`::Granted request >Thread-171::DEBUG::2014-08-22 09:13:56,603::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6e828223-4b11-430f-bd68-c84ff812f9a5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:13:56,603::task::993::Storage.TaskManager.Task::(_decref) Task=`6e828223-4b11-430f-bd68-c84ff812f9a5`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:13:56,605::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:56,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e828223-4b11-430f-bd68-c84ff812f9a5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:13:56,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e828223-4b11-430f-bd68-c84ff812f9a5`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:13:56,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:13:56,605::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:13:56,605::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:13:56,605::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:13:56,605::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:13:56,606::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:13:56,606::task::993::Storage.TaskManager.Task::(_decref) Task=`6e828223-4b11-430f-bd68-c84ff812f9a5`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:56,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:56,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:13:56,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:13:57,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:57,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115431 s, 88.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:57,219::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:57,220::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:57,220::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:57,221::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:57,221::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:57,222::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:57,223::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:57,223::__init__::225::IOProcess::(_processLogs) (190842) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:57,224::__init__::225::IOProcess::(_processLogs) (190842) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:57,224::__init__::225::IOProcess::(_processLogs) (190842) Building response >Thread-14::DEBUG::2014-08-22 09:13:57,225::__init__::225::IOProcess::(_processLogs) (190842) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:57,225::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:57,226::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:13:57,229::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:13:57,258::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189958 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:13:57,260::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:13:57,261::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:13:57,261::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:13:57,262::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:13:57,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:13:57,263::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:13:57,263::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:13:57,264::__init__::225::IOProcess::(_processLogs) (190843) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:13:57,264::__init__::225::IOProcess::(_processLogs) (190843) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:13:57,265::__init__::225::IOProcess::(_processLogs) (190843) Building response >Thread-14::DEBUG::2014-08-22 09:13:57,265::__init__::225::IOProcess::(_processLogs) (190843) Queuing response >Thread-14::DEBUG::2014-08-22 09:13:57,266::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:13:57,267::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:13:59,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:13:59,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117953 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:01,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:01,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117829 s, 86.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:01,417::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:01,418::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:01,418::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:01,419::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:01,420::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:01,420::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:01,421::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:01,421::__init__::225::IOProcess::(_processLogs) (190844) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:01,422::__init__::225::IOProcess::(_processLogs) (190844) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:01,423::__init__::225::IOProcess::(_processLogs) (190844) Building response >Thread-14::DEBUG::2014-08-22 09:14:01,423::__init__::225::IOProcess::(_processLogs) (190844) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:01,424::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:01,424::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:14:01,428::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:14:01,457::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000190078 s, 3.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:01,459::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:01,460::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:01,460::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:01,461::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:01,461::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:01,462::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:01,463::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:01,463::__init__::225::IOProcess::(_processLogs) (190845) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:01,464::__init__::225::IOProcess::(_processLogs) (190845) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:01,464::__init__::225::IOProcess::(_processLogs) (190845) Building response >Thread-14::DEBUG::2014-08-22 09:14:01,465::__init__::225::IOProcess::(_processLogs) (190845) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:01,465::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:01,466::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:14:01,470::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:01,471::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:14:01,471::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:01,472::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:01,472::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:01,473::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:01,474::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:01,474::__init__::225::IOProcess::(_processLogs) (190846) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:01,475::__init__::225::IOProcess::(_processLogs) (190846) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:01,475::__init__::225::IOProcess::(_processLogs) (190846) Building response >Thread-14::DEBUG::2014-08-22 09:14:01,476::__init__::225::IOProcess::(_processLogs) (190846) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:01,477::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:01,477::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:01,479::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:01,480::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:14:01,480::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:01,481::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:01,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:01,482::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:01,483::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:01,483::__init__::225::IOProcess::(_processLogs) (190847) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:01,484::__init__::225::IOProcess::(_processLogs) (190847) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:01,485::__init__::225::IOProcess::(_processLogs) (190847) Building response >Thread-14::DEBUG::2014-08-22 09:14:01,485::__init__::225::IOProcess::(_processLogs) (190847) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:01,486::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:01,486::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:01,490::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:01,490::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:14:01,491::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:01,491::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:01,492::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:01,492::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:01,493::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:01,493::__init__::225::IOProcess::(_processLogs) (190848) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:01,494::__init__::225::IOProcess::(_processLogs) (190848) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:01,495::__init__::225::IOProcess::(_processLogs) (190848) Building response >Thread-14::DEBUG::2014-08-22 09:14:01,495::__init__::225::IOProcess::(_processLogs) (190848) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:01,496::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:01,496::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:01,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:01,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:01,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:03,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:03,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01158 s, 88.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:04,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`e81348d1-bd21-483e-a348-b8225b4adfb5`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:04,151::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:14:04,151::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189958', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000190078', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:04,151::task::1191::Storage.TaskManager.Task::(prepare) Task=`e81348d1-bd21-483e-a348-b8225b4adfb5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189958', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000190078', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:04,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`e81348d1-bd21-483e-a348-b8225b4adfb5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:04,151::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:04,151::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:04,151::task::993::Storage.TaskManager.Task::(_decref) Task=`e81348d1-bd21-483e-a348-b8225b4adfb5`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:14:04,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:05,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:05,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116709 s, 87.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:06,655::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-13::DEBUG::2014-08-22 09:14:06,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4d2f980-6792-476b-95c8-958110c7b6ac`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:06,656::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:14:06,656::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:06,656::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4d2f980-6792-476b-95c8-958110c7b6ac`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:06,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4d2f980-6792-476b-95c8-958110c7b6ac`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:06,656::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:06,656::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:06,656::task::993::Storage.TaskManager.Task::(_decref) Task=`d4d2f980-6792-476b-95c8-958110c7b6ac`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:14:06,681::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-171::DEBUG::2014-08-22 09:14:06,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`b78ec7c0-c7e9-4f04-b2c1-dab08cc002ce`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:14:06,681::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:14:06,682::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0f0b3a78-a2e3-4d60-abce-e5a34c7473e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:14:06,682::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:14:06,682::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:14:06,682::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0f0b3a78-a2e3-4d60-abce-e5a34c7473e8`::Granted request >Thread-171::DEBUG::2014-08-22 09:14:06,682::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b78ec7c0-c7e9-4f04-b2c1-dab08cc002ce`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:14:06,682::task::993::Storage.TaskManager.Task::(_decref) Task=`b78ec7c0-c7e9-4f04-b2c1-dab08cc002ce`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:14:06,686::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:06,686::task::1191::Storage.TaskManager.Task::(prepare) Task=`b78ec7c0-c7e9-4f04-b2c1-dab08cc002ce`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:06,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`b78ec7c0-c7e9-4f04-b2c1-dab08cc002ce`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:14:06,686::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:14:06,686::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:14:06,686::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:14:06,687::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:14:06,687::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:14:06,687::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:14:06,687::task::993::Storage.TaskManager.Task::(_decref) Task=`b78ec7c0-c7e9-4f04-b2c1-dab08cc002ce`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:06,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:06,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:06,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-91::DEBUG::2014-08-22 09:14:07,270::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,272::__init__::225::IOProcess::(_processLogs) (190849) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) (190849) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) Message size is 145 >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) (190850) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:14:07,273::__init__::225::IOProcess::(_processLogs) (190850) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:14:07,274::__init__::225::IOProcess::(_processLogs) DEBUG|(190850) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,274::__init__::225::IOProcess::(_processLogs) (190850) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,274::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,274::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 195 >Thread-14::DEBUG::2014-08-22 09:14:07,274::__init__::225::IOProcess::(_processLogs) DEBUG|(190849) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,275::__init__::225::IOProcess::(_processLogs) (190849) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,275::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-91::DEBUG::2014-08-22 09:14:07,277::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-14::DEBUG::2014-08-22 09:14:07,277::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,277::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-14::DEBUG::2014-08-22 09:14:07,277::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,278::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,278::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,278::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,278::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,278::__init__::225::IOProcess::(_processLogs) (190851) Finding callback 'touch'... >Thread-14::DEBUG::2014-08-22 09:14:07,278::__init__::225::IOProcess::(_processLogs) (190851) Got request for method 'touch' >Thread-14::DEBUG::2014-08-22 09:14:07,307::__init__::225::IOProcess::(_processLogs) DEBUG|(190851) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,307::__init__::225::IOProcess::(_processLogs) (190851) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,308::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,308::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-91::DEBUG::2014-08-22 09:14:07,308::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-22 09:14:07,308::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,308::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-14::DEBUG::2014-08-22 09:14:07,308::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) (190852) Finding callback 'readfile'... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) (190852) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) DEBUG|(190852) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) (190852) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,309::__init__::225::IOProcess::(_processLogs) Sending response sized 606 >Thread-91::DEBUG::2014-08-22 09:14:07,310::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-14::DEBUG::2014-08-22 09:14:07,310::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,310::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-14::DEBUG::2014-08-22 09:14:07,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,310::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,310::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) (190853) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) (190853) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) (190853) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) (190853) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,311::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) (190854) Finding callback 'glob'... >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) (190854) Got request for method 'glob' >Thread-14::DEBUG::2014-08-22 09:14:07,312::__init__::225::IOProcess::(_processLogs) DEBUG|(190854) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,313::__init__::225::IOProcess::(_processLogs) (190854) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,313::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-91::DEBUG::2014-08-22 09:14:07,313::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-91::INFO::2014-08-22 09:14:07,313::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-91::INFO::2014-08-22 09:14:07,313::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-14::DEBUG::2014-08-22 09:14:07,313::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) (190855) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) (190855) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) (190855) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) (190855) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,314::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,315::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:14:07,315::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:14:07,345::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000221495 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:07,347::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:07,348::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:07,348::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:07,348::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) (190856) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) (190856) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) (190856) Building response >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) (190856) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:07,349::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:14:07,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:07,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228266 s, 44.9 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:09,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:09,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118226 s, 86.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:11,499::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:11,500::__init__::225::IOProcess::(_processLogs) (190857) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:11,500::__init__::225::IOProcess::(_processLogs) (190857) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:11,500::__init__::225::IOProcess::(_processLogs) DEBUG|(190857) Building response >Thread-14::DEBUG::2014-08-22 09:14:11,500::__init__::225::IOProcess::(_processLogs) (190857) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:11,500::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:11,500::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:14:11,501::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:14:11,529::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000244896 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:11,530::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:11,531::__init__::225::IOProcess::(_processLogs) (190858) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:11,531::__init__::225::IOProcess::(_processLogs) (190858) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:11,531::__init__::225::IOProcess::(_processLogs) DEBUG|(190858) Building response >Thread-14::DEBUG::2014-08-22 09:14:11,531::__init__::225::IOProcess::(_processLogs) (190858) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:11,531::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:11,531::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:11,532::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) (190859) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) (190859) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) (190859) Building response >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) (190859) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:11,533::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) (190860) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) (190860) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) (190860) Building response >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) (190860) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:11,534::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:11,535::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) (190861) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) (190861) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) (190861) Building response >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) (190861) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:11,536::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:11,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:11,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:11,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:11,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:11,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117569 s, 87.1 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:14:13,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1613d3a-8d53-416c-82ab-55a52ac7c223`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:14:13,608::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:14:13,608::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:13,608::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:14:13,608::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) (190862) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) (190862) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) DEBUG|(190862) Building response >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) (190862) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:13,609::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:14:13,610::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:13,610::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:14:13,610::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:13,610::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) (190863) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) (190863) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) (190863) Building response >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) (190863) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:13,611::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:14:13,612::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:14:13,612::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1613d3a-8d53-416c-82ab-55a52ac7c223`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:14:13,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1613d3a-8d53-416c-82ab-55a52ac7c223`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:14:13,612::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:14:13,612::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:14:13,612::task::993::Storage.TaskManager.Task::(_decref) Task=`b1613d3a-8d53-416c-82ab-55a52ac7c223`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:14:14,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:14,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156453 s, 65.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:16,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:16,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120566 s, 84.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:16,732::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:14:16,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e98cbc3-295d-45e4-adf7-15d5865f4553`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:16,733::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:14:16,733::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:16,733::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e98cbc3-295d-45e4-adf7-15d5865f4553`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:16,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e98cbc3-295d-45e4-adf7-15d5865f4553`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:16,733::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:16,733::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:16,734::task::993::Storage.TaskManager.Task::(_decref) Task=`2e98cbc3-295d-45e4-adf7-15d5865f4553`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:14:16,758::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:14:16,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`da80c3c1-1d38-4f81-881a-0ae7da693eb5`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:14:16,758::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:14:16,758::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`54d4dfc4-fa40-4220-97bc-30389c6bfe91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:14:16,758::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:14:16,759::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:14:16,759::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`54d4dfc4-fa40-4220-97bc-30389c6bfe91`::Granted request >Thread-171::DEBUG::2014-08-22 09:14:16,759::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`da80c3c1-1d38-4f81-881a-0ae7da693eb5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:14:16,759::task::993::Storage.TaskManager.Task::(_decref) Task=`da80c3c1-1d38-4f81-881a-0ae7da693eb5`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:14:16,761::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:16,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`da80c3c1-1d38-4f81-881a-0ae7da693eb5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:16,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`da80c3c1-1d38-4f81-881a-0ae7da693eb5`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:14:16,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:14:16,761::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:14:16,762::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:14:16,762::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:14:16,762::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:14:16,762::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:14:16,762::task::993::Storage.TaskManager.Task::(_decref) Task=`da80c3c1-1d38-4f81-881a-0ae7da693eb5`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:16,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:16,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:16,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:14:17,350::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) (190864) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:17,351::__init__::225::IOProcess::(_processLogs) (190864) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:17,352::__init__::225::IOProcess::(_processLogs) DEBUG|(190864) Building response >Thread-14::DEBUG::2014-08-22 09:14:17,352::__init__::225::IOProcess::(_processLogs) (190864) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:17,352::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:14:17,352::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:14:17,352::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:14:17,381::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000261776 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) (190865) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) (190865) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:17,382::__init__::225::IOProcess::(_processLogs) (190865) Building response >Thread-14::DEBUG::2014-08-22 09:14:17,383::__init__::225::IOProcess::(_processLogs) (190865) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:17,383::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:17,383::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:14:18,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:18,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114911 s, 89.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:19,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`7826ddbc-a57b-42a9-9443-88d50553a301`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:19,403::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:14:19,405::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000261776', 'lastCheck': '2.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000244896', 'lastCheck': '7.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:19,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`7826ddbc-a57b-42a9-9443-88d50553a301`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000261776', 'lastCheck': '2.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000244896', 'lastCheck': '7.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:19,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`7826ddbc-a57b-42a9-9443-88d50553a301`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:19,407::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:19,407::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:19,408::task::993::Storage.TaskManager.Task::(_decref) Task=`7826ddbc-a57b-42a9-9443-88d50553a301`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:14:19,452::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:20,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:20,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214873 s, 47.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:21,537::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:21,537::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:21,537::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:21,537::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) (190866) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) (190866) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) (190866) Building response >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) (190866) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:21,538::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:14:21,539::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:14:21,567::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00025307 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) (190867) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) (190867) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) (190867) Building response >Thread-14::DEBUG::2014-08-22 09:14:21,568::__init__::225::IOProcess::(_processLogs) (190867) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:21,569::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:21,569::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:14:21,569::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:21,570::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:14:21,570::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:21,570::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:21,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:21,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:21,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:21,571::__init__::225::IOProcess::(_processLogs) (190868) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:21,571::__init__::225::IOProcess::(_processLogs) (190868) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:21,571::__init__::225::IOProcess::(_processLogs) (190868) Building response >Thread-14::DEBUG::2014-08-22 09:14:21,571::__init__::225::IOProcess::(_processLogs) (190868) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:21,572::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:21,572::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:21,572::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:21,572::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:14:21,572::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:21,572::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) (190869) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) (190869) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) (190869) Building response >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) (190869) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:21,573::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:21,574::__init__::225::IOProcess::(_processLogs) (190870) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:21,575::__init__::225::IOProcess::(_processLogs) (190870) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:21,576::__init__::225::IOProcess::(_processLogs) (190870) Building response >Thread-14::DEBUG::2014-08-22 09:14:21,591::__init__::225::IOProcess::(_processLogs) (190870) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:21,592::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:21,592::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:21,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:21,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:21,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:22,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:22,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118774 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:24,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:24,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117082 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:26,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:26,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119081 s, 86.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:26,809::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:14:26,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d0f2769-c44b-4320-bd40-e82662ecb82d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:26,810::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:14:26,810::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:26,810::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d0f2769-c44b-4320-bd40-e82662ecb82d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:26,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d0f2769-c44b-4320-bd40-e82662ecb82d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:26,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:26,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:26,810::task::993::Storage.TaskManager.Task::(_decref) Task=`9d0f2769-c44b-4320-bd40-e82662ecb82d`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:14:26,835::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:14:26,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b328ecc-38e7-4861-aeb6-f9aa6753028a`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:14:26,835::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:14:26,835::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3ad624d0-713c-4799-9692-e70c0269c3ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:14:26,835::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:14:26,835::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:14:26,836::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3ad624d0-713c-4799-9692-e70c0269c3ae`::Granted request >Thread-171::DEBUG::2014-08-22 09:14:26,837::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7b328ecc-38e7-4861-aeb6-f9aa6753028a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:14:26,837::task::993::Storage.TaskManager.Task::(_decref) Task=`7b328ecc-38e7-4861-aeb6-f9aa6753028a`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:14:26,839::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:26,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b328ecc-38e7-4861-aeb6-f9aa6753028a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:26,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b328ecc-38e7-4861-aeb6-f9aa6753028a`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:14:26,839::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:14:26,840::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:14:26,840::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:14:26,840::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:14:26,840::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:14:26,840::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:14:26,840::task::993::Storage.TaskManager.Task::(_decref) Task=`7b328ecc-38e7-4861-aeb6-f9aa6753028a`::ref 0 aborting False >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:26,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:26,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:26,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:14:27,384::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:27,384::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:27,384::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:27,384::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:27,384::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:27,384::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) (190871) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) (190871) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) (190871) Building response >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) (190871) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:27,385::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:14:27,386::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:14:27,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000290741 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:27,414::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:27,414::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:27,414::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) (190872) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) (190872) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) (190872) Building response >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) (190872) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:27,415::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:14:28,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:28,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170403 s, 60.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:30,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:30,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011462 s, 89.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:31,593::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) (190873) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) (190873) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) (190873) Building response >Thread-14::DEBUG::2014-08-22 09:14:31,594::__init__::225::IOProcess::(_processLogs) (190873) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:31,595::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:31,595::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:14:31,595::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:14:31,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000233208 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:31,624::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:31,624::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:31,624::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:31,624::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:31,624::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) (190874) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) (190874) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) (190874) Building response >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) (190874) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:31,625::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:14:31,626::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:31,626::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:14:31,626::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:31,626::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:31,626::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:31,626::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) (190875) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) (190875) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) (190875) Building response >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) (190875) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:31,627::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) (190876) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) (190876) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) (190876) Building response >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) (190876) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:31,628::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:31,629::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:31,629::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:31,629::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:14:31,629::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:31,629::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:31,629::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) (190877) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) (190877) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) (190877) Building response >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) (190877) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:31,630::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:31,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:31,914::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:31,914::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:32,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:32,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012116 s, 84.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:34,661::task::595::Storage.TaskManager.Task::(_updateState) Task=`94697ce0-d843-4be1-99e9-6fa38cff49e9`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:34,661::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:14:34,662::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000290741', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000233208', 'lastCheck': '3.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:34,662::task::1191::Storage.TaskManager.Task::(prepare) Task=`94697ce0-d843-4be1-99e9-6fa38cff49e9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000290741', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000233208', 'lastCheck': '3.0', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:34,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`94697ce0-d843-4be1-99e9-6fa38cff49e9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:34,662::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:34,662::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:34,662::task::993::Storage.TaskManager.Task::(_decref) Task=`94697ce0-d843-4be1-99e9-6fa38cff49e9`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:14:34,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:34,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:34,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117261 s, 87.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:36,902::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:14:36,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d9193f1-c709-4ca3-aec2-c0ab364aa8be`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:36,903::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:14:36,903::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:36,903::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d9193f1-c709-4ca3-aec2-c0ab364aa8be`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:36,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d9193f1-c709-4ca3-aec2-c0ab364aa8be`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:36,903::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:36,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:36,903::task::993::Storage.TaskManager.Task::(_decref) Task=`9d9193f1-c709-4ca3-aec2-c0ab364aa8be`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:14:36,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:36,934::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-171::DEBUG::2014-08-22 09:14:36,935::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:14:36,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f78763e-6a06-40b6-bee3-8aad9aad5043`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:14:36,937::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:36,937::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-171::DEBUG::2014-08-22 09:14:36,939::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9fb31026-3e24-4165-9398-d7dbe348abd5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:14:36,939::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:36,940::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-171::DEBUG::2014-08-22 09:14:36,940::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:14:36,941::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9fb31026-3e24-4165-9398-d7dbe348abd5`::Granted request >Thread-171::DEBUG::2014-08-22 09:14:36,942::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6f78763e-6a06-40b6-bee3-8aad9aad5043`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:14:36,942::task::993::Storage.TaskManager.Task::(_decref) Task=`6f78763e-6a06-40b6-bee3-8aad9aad5043`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:14:36,952::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:36,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f78763e-6a06-40b6-bee3-8aad9aad5043`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:36,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f78763e-6a06-40b6-bee3-8aad9aad5043`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:14:36,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:14:36,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:14:36,979::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Dummy-170::DEBUG::2014-08-22 09:14:36,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215997 s, 47.4 MB/s\n'; <rc> = 0 >Thread-171::DEBUG::2014-08-22 09:14:36,981::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:14:36,981::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:14:36,981::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:14:36,982::task::993::Storage.TaskManager.Task::(_decref) Task=`6f78763e-6a06-40b6-bee3-8aad9aad5043`::ref 0 aborting False >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) (190878) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) (190878) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) DEBUG|(190878) Building response >Thread-14::DEBUG::2014-08-22 09:14:37,417::__init__::225::IOProcess::(_processLogs) (190878) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:37,418::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:37,418::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:14:37,418::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:14:37,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000251775 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:37,447::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:37,448::__init__::225::IOProcess::(_processLogs) (190879) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:37,448::__init__::225::IOProcess::(_processLogs) (190879) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:37,448::__init__::225::IOProcess::(_processLogs) (190879) Building response >Thread-14::DEBUG::2014-08-22 09:14:37,448::__init__::225::IOProcess::(_processLogs) (190879) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:37,448::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:37,448::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:14:38,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:39,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116908 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:41,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:41,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118821 s, 86.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:41,631::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:41,631::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:41,631::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:41,631::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) (190880) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) (190880) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) DEBUG|(190880) Building response >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) (190880) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:41,632::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:14:41,633::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:14:41,661::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000167697 s, 3.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:41,662::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:41,662::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:41,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:41,662::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) (190881) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) (190881) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) (190881) Building response >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) (190881) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:41,663::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:14:41,664::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:41,664::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:14:41,664::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:41,664::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:41,664::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) (190882) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) (190882) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) (190882) Building response >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) (190882) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:41,665::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) (190883) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) (190883) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) (190883) Building response >Thread-14::DEBUG::2014-08-22 09:14:41,666::__init__::225::IOProcess::(_processLogs) (190883) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:41,667::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) (190884) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) (190884) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) (190884) Building response >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) (190884) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:41,668::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:41,947::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:41,948::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:41,948::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:43,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:43,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120829 s, 84.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:45,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:45,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118052 s, 86.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:46,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:46,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:46,968::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:14:47,066::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:14:47,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a505acb-6f03-4e49-85a3-c6bcd9be8a24`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:47,067::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:14:47,067::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:47,067::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a505acb-6f03-4e49-85a3-c6bcd9be8a24`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:47,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a505acb-6f03-4e49-85a3-c6bcd9be8a24`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:47,067::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:47,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:47,068::task::993::Storage.TaskManager.Task::(_decref) Task=`5a505acb-6f03-4e49-85a3-c6bcd9be8a24`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:14:47,172::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:14:47,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`73c4297d-c3d2-473a-ba9f-03e4d5320ee4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:14:47,173::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:14:47,173::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a758f8d8-bc6f-4172-b807-65f9354d478a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:14:47,173::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:14:47,173::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:14:47,173::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a758f8d8-bc6f-4172-b807-65f9354d478a`::Granted request >Thread-171::DEBUG::2014-08-22 09:14:47,174::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`73c4297d-c3d2-473a-ba9f-03e4d5320ee4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:14:47,174::task::993::Storage.TaskManager.Task::(_decref) Task=`73c4297d-c3d2-473a-ba9f-03e4d5320ee4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:14:47,176::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:47,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`73c4297d-c3d2-473a-ba9f-03e4d5320ee4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:47,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`73c4297d-c3d2-473a-ba9f-03e4d5320ee4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:14:47,176::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:14:47,176::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:14:47,176::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:14:47,177::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:14:47,177::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:14:47,177::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:14:47,177::task::993::Storage.TaskManager.Task::(_decref) Task=`73c4297d-c3d2-473a-ba9f-03e4d5320ee4`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:14:47,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:47,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121239 s, 84.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:47,449::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:47,449::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:47,449::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) (190885) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) (190885) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) (190885) Building response >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) (190885) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:47,450::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:14:47,451::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:14:47,479::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000225931 s, 1.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) (190886) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:47,480::__init__::225::IOProcess::(_processLogs) (190886) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:47,481::__init__::225::IOProcess::(_processLogs) (190886) Building response >Thread-14::DEBUG::2014-08-22 09:14:47,481::__init__::225::IOProcess::(_processLogs) (190886) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:47,481::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:47,481::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:14:49,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:49,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217952 s, 47.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:14:49,864::task::595::Storage.TaskManager.Task::(_updateState) Task=`d45e62fd-7ce7-409e-a1ad-dd3906f67dcf`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:49,865::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:14:49,866::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000225931', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000167697', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:49,866::task::1191::Storage.TaskManager.Task::(prepare) Task=`d45e62fd-7ce7-409e-a1ad-dd3906f67dcf`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000225931', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000167697', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:14:49,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`d45e62fd-7ce7-409e-a1ad-dd3906f67dcf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:49,867::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:49,867::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:49,868::task::993::Storage.TaskManager.Task::(_decref) Task=`d45e62fd-7ce7-409e-a1ad-dd3906f67dcf`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:14:49,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:51,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:51,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119997 s, 85.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:51,669::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:51,669::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:51,669::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:51,670::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:51,670::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:51,670::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:51,670::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:51,670::__init__::225::IOProcess::(_processLogs) (190887) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:51,670::__init__::225::IOProcess::(_processLogs) (190887) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:51,671::__init__::225::IOProcess::(_processLogs) (190887) Building response >Thread-14::DEBUG::2014-08-22 09:14:51,671::__init__::225::IOProcess::(_processLogs) (190887) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:51,671::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:14:51,671::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:14:51,672::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:14:51,701::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000217962 s, 2.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:51,702::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:51,702::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:14:51,702::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:51,702::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:51,702::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) (190888) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) (190888) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) DEBUG|(190888) Building response >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) (190888) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:51,703::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:14:51,704::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:51,704::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:14:51,704::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:51,704::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:51,704::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) (190889) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) (190889) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) (190889) Building response >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) (190889) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:51,705::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) (190890) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) (190890) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) DEBUG|(190890) Building response >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) (190890) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:51,706::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:51,707::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:14:51,707::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:51,707::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:14:51,707::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:51,707::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:51,707::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) (190891) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) (190891) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) (190891) Building response >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) (190891) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:51,708::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:51,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:51,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:51,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:14:53,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:53,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121976 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:55,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:55,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116653 s, 87.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:56,980::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:56,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:14:56,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:14:57,350::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:14:57,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`0adf2a75-244b-48d8-9881-a5fbf14e99aa`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:14:57,351::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:14:57,351::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:57,351::task::1191::Storage.TaskManager.Task::(prepare) Task=`0adf2a75-244b-48d8-9881-a5fbf14e99aa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:14:57,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`0adf2a75-244b-48d8-9881-a5fbf14e99aa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:14:57,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:14:57,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:14:57,352::task::993::Storage.TaskManager.Task::(_decref) Task=`0adf2a75-244b-48d8-9881-a5fbf14e99aa`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:14:57,388::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:14:57,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ad8f866-d63e-4fd2-89b6-d9cdb806e2a4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:14:57,388::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:14:57,389::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`aa98906c-73e3-436b-85ae-93007add3a04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:14:57,389::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:14:57,389::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:14:57,389::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`aa98906c-73e3-436b-85ae-93007add3a04`::Granted request >Thread-171::DEBUG::2014-08-22 09:14:57,389::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6ad8f866-d63e-4fd2-89b6-d9cdb806e2a4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:14:57,389::task::993::Storage.TaskManager.Task::(_decref) Task=`6ad8f866-d63e-4fd2-89b6-d9cdb806e2a4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:14:57,391::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:57,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ad8f866-d63e-4fd2-89b6-d9cdb806e2a4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:14:57,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ad8f866-d63e-4fd2-89b6-d9cdb806e2a4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:14:57,392::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:14:57,392::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:14:57,392::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:14:57,392::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:14:57,392::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:14:57,392::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:14:57,392::task::993::Storage.TaskManager.Task::(_decref) Task=`6ad8f866-d63e-4fd2-89b6-d9cdb806e2a4`::ref 0 aborting False >Thread-14::DEBUG::2014-08-22 09:14:57,482::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:57,482::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:57,482::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:57,482::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:57,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:57,482::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) (190892) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) (190892) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) (190892) Building response >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) (190892) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:14:57,483::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:14:57,484::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:14:57,511::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000250788 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:14:57,512::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:14:57,512::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:14:57,512::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) (190893) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) (190893) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) (190893) Building response >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) (190893) Queuing response >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:14:57,513::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:14:57,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:57,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123413 s, 83.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:14:59,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:14:59,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120332 s, 85.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:01,709::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:01,709::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:01,709::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:01,709::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) (190894) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) (190894) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) DEBUG|(190894) Building response >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) (190894) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:01,710::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:15:01,711::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:15:01,739::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000240528 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:01,740::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:01,740::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:01,740::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:01,740::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:01,740::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) (190895) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) (190895) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) (190895) Building response >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) (190895) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:01,741::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:15:01,742::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:01,742::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:15:01,742::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:01,742::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:01,742::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) (190896) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) (190896) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) (190896) Building response >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) (190896) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:01,743::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) (190897) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) (190897) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) (190897) Building response >Thread-14::DEBUG::2014-08-22 09:15:01,744::__init__::225::IOProcess::(_processLogs) (190897) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:01,745::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:01,745::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:01,745::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:01,745::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:15:01,745::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:01,745::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) (190898) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) (190898) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) (190898) Building response >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) (190898) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:01,746::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:15:01,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:01,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011937 s, 85.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:01,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:01,990::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:01,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:04,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:04,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011845 s, 86.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:15:05,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`78d67428-7b20-422f-8c5b-6efe0697eac6`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:05,307::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:15:05,307::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000250788', 'lastCheck': '7.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000240528', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:05,307::task::1191::Storage.TaskManager.Task::(prepare) Task=`78d67428-7b20-422f-8c5b-6efe0697eac6`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000250788', 'lastCheck': '7.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000240528', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:05,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`78d67428-7b20-422f-8c5b-6efe0697eac6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:05,307::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:05,307::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:05,307::task::993::Storage.TaskManager.Task::(_decref) Task=`78d67428-7b20-422f-8c5b-6efe0697eac6`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:15:05,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:06,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:06,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115848 s, 88.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:06,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:07,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:07,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:15:07,490::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-13::DEBUG::2014-08-22 09:15:07,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4d175e1-9409-4b58-98f9-b955cc434e48`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:07,490::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:15:07,491::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:07,491::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4d175e1-9409-4b58-98f9-b955cc434e48`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:07,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4d175e1-9409-4b58-98f9-b955cc434e48`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:07,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:07,491::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:07,491::task::993::Storage.TaskManager.Task::(_decref) Task=`a4d175e1-9409-4b58-98f9-b955cc434e48`::ref 0 aborting False >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) DEBUG|Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) (190899) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:07,515::__init__::225::IOProcess::(_processLogs) (190899) Got request for method 'statvfs' >Thread-171::DEBUG::2014-08-22 09:15:07,516::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-171::DEBUG::2014-08-22 09:15:07,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`c12167bc-9a5f-44a3-906e-df76a3185e81`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:15:07,516::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:15:07,517::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a4f0bdbe-5b1f-486e-895d-c52c64301a82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:15:07,517::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-14::DEBUG::2014-08-22 09:15:07,517::__init__::225::IOProcess::(_processLogs) DEBUG|(190899) Building response >Thread-171::DEBUG::2014-08-22 09:15:07,518::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-14::DEBUG::2014-08-22 09:15:07,518::__init__::225::IOProcess::(_processLogs) (190899) Queuing response >Thread-171::DEBUG::2014-08-22 09:15:07,518::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a4f0bdbe-5b1f-486e-895d-c52c64301a82`::Granted request >Thread-14::DEBUG::2014-08-22 09:15:07,519::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:07,520::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-171::DEBUG::2014-08-22 09:15:07,520::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c12167bc-9a5f-44a3-906e-df76a3185e81`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:15:07,521::task::993::Storage.TaskManager.Task::(_decref) Task=`c12167bc-9a5f-44a3-906e-df76a3185e81`::ref 1 aborting False >Thread-91::DEBUG::2014-08-22 09:15:07,524::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:15:07,553::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000152179 s, 2.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:07,556::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:07,557::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:07,558::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:07,559::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:07,560::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:07,561::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:07,562::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:07,563::__init__::225::IOProcess::(_processLogs) (190900) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:07,564::__init__::225::IOProcess::(_processLogs) (190900) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:07,565::__init__::225::IOProcess::(_processLogs) DEBUG|(190900) Building response >Thread-14::DEBUG::2014-08-22 09:15:07,566::__init__::225::IOProcess::(_processLogs) (190900) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:07,567::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:07,568::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-171::INFO::2014-08-22 09:15:07,568::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:07,569::task::1191::Storage.TaskManager.Task::(prepare) Task=`c12167bc-9a5f-44a3-906e-df76a3185e81`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:07,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`c12167bc-9a5f-44a3-906e-df76a3185e81`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:15:07,571::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:15:07,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:15:07,572::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:15:07,572::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:15:07,572::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:15:07,572::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:15:07,573::task::993::Storage.TaskManager.Task::(_decref) Task=`c12167bc-9a5f-44a3-906e-df76a3185e81`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:08,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:08,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121419 s, 84.3 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:10,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:10,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116569 s, 87.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:11,747::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:11,747::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) (190901) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) (190901) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) (190901) Building response >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) (190901) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:11,748::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:15:11,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:15:11,777::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000297768 s, 1.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:11,778::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:11,778::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:11,778::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:11,778::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:11,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:11,778::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) (190902) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) (190902) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) (190902) Building response >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) (190902) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:11,779::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:11,780::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) (190903) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) (190903) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) (190903) Building response >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) (190903) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:11,781::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) (190904) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) (190904) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) (190904) Building response >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) (190904) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:11,782::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:11,783::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) (190905) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) (190905) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) (190905) Building response >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) (190905) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:11,784::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:12,006::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:12,006::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:12,007::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:12,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:12,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181003 s, 56.6 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:15:13,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`50d0bd02-274f-46ac-b6d9-d061c6d7ac04`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:15:13,703::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:15:13,704::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:13,704::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:15:13,704::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:13,704::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) (190906) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) (190906) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) DEBUG|(190906) Building response >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) (190906) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:15:13,705::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:13,706::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:13,707::__init__::225::IOProcess::(_processLogs) (190907) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:15:13,707::__init__::225::IOProcess::(_processLogs) (190907) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:15:13,707::__init__::225::IOProcess::(_processLogs) (190907) Building response >Thread-14::DEBUG::2014-08-22 09:15:13,707::__init__::225::IOProcess::(_processLogs) (190907) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:13,707::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:13,707::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:15:13,707::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:15:13,708::task::1191::Storage.TaskManager.Task::(prepare) Task=`50d0bd02-274f-46ac-b6d9-d061c6d7ac04`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:15:13,708::task::595::Storage.TaskManager.Task::(_updateState) Task=`50d0bd02-274f-46ac-b6d9-d061c6d7ac04`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:15:13,708::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:15:13,708::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:15:13,708::task::993::Storage.TaskManager.Task::(_decref) Task=`50d0bd02-274f-46ac-b6d9-d061c6d7ac04`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:14,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:14,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121951 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:16,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:16,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118134 s, 86.7 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:17,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:17,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:17,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) (190908) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:17,573::__init__::225::IOProcess::(_processLogs) (190908) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:17,574::__init__::225::IOProcess::(_processLogs) (190908) Building response >Thread-14::DEBUG::2014-08-22 09:15:17,574::__init__::225::IOProcess::(_processLogs) (190908) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:17,574::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:17,574::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:15:17,574::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:15:17,604::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000258941 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) (190909) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:17,607::__init__::225::IOProcess::(_processLogs) (190909) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:17,608::__init__::225::IOProcess::(_processLogs) (190909) Building response >Thread-14::DEBUG::2014-08-22 09:15:17,608::__init__::225::IOProcess::(_processLogs) (190909) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:17,608::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:17,608::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:15:17,646::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:15:17,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`178a1866-118c-4f68-a86f-2dc68a548359`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:17,646::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:15:17,647::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:17,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`178a1866-118c-4f68-a86f-2dc68a548359`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:17,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`178a1866-118c-4f68-a86f-2dc68a548359`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:17,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:17,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:17,647::task::993::Storage.TaskManager.Task::(_decref) Task=`178a1866-118c-4f68-a86f-2dc68a548359`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:15:17,745::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:15:17,746::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bdebecc-146b-4ba8-8552-acfc2f533efe`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:15:17,746::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:15:17,746::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b7254e8a-350e-4394-b854-90fd7b67bdf8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:15:17,746::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:15:17,746::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:15:17,746::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b7254e8a-350e-4394-b854-90fd7b67bdf8`::Granted request >Thread-171::DEBUG::2014-08-22 09:15:17,747::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9bdebecc-146b-4ba8-8552-acfc2f533efe`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:15:17,747::task::993::Storage.TaskManager.Task::(_decref) Task=`9bdebecc-146b-4ba8-8552-acfc2f533efe`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:15:17,749::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:17,749::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bdebecc-146b-4ba8-8552-acfc2f533efe`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:17,749::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bdebecc-146b-4ba8-8552-acfc2f533efe`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:15:17,749::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:15:17,749::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:15:17,749::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:15:17,749::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:15:17,750::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:15:17,750::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:15:17,750::task::993::Storage.TaskManager.Task::(_decref) Task=`9bdebecc-146b-4ba8-8552-acfc2f533efe`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:18,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:18,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212933 s, 48.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:15:20,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3db99cf-c862-437a-a6cd-90c9f43e40ac`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:20,619::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:15:20,619::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000258941', 'lastCheck': '3.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000297768', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:20,619::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3db99cf-c862-437a-a6cd-90c9f43e40ac`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000258941', 'lastCheck': '3.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000297768', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:20,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3db99cf-c862-437a-a6cd-90c9f43e40ac`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:20,620::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:20,620::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:20,620::task::993::Storage.TaskManager.Task::(_decref) Task=`d3db99cf-c862-437a-a6cd-90c9f43e40ac`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:15:20,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:20,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:20,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183732 s, 55.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:21,785::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:21,785::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:21,785::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) (190910) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) (190910) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) (190910) Building response >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) (190910) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:21,786::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:15:21,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:15:21,815::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000224277 s, 2.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:21,816::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:21,817::__init__::225::IOProcess::(_processLogs) (190911) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:21,817::__init__::225::IOProcess::(_processLogs) (190911) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:21,817::__init__::225::IOProcess::(_processLogs) (190911) Building response >Thread-14::DEBUG::2014-08-22 09:15:21,817::__init__::225::IOProcess::(_processLogs) (190911) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:21,817::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:21,817::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:21,818::__init__::225::IOProcess::(_processLogs) (190912) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:21,819::__init__::225::IOProcess::(_processLogs) (190912) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:21,819::__init__::225::IOProcess::(_processLogs) (190912) Building response >Thread-14::DEBUG::2014-08-22 09:15:21,819::__init__::225::IOProcess::(_processLogs) (190912) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:21,819::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:21,819::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:21,819::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) (190913) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) (190913) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) (190913) Building response >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) (190913) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:21,820::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:21,821::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:21,822::__init__::225::IOProcess::(_processLogs) (190914) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:21,822::__init__::225::IOProcess::(_processLogs) (190914) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:21,822::__init__::225::IOProcess::(_processLogs) (190914) Building response >Thread-14::DEBUG::2014-08-22 09:15:21,822::__init__::225::IOProcess::(_processLogs) (190914) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:21,822::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:21,822::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:22,019::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:22,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:22,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:22,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:22,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119843 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:24,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:24,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118003 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:26,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:27,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120284 s, 85.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:27,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:27,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:27,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:15:27,616::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) (190915) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:27,617::__init__::225::IOProcess::(_processLogs) (190915) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:27,618::__init__::225::IOProcess::(_processLogs) (190915) Building response >Thread-14::DEBUG::2014-08-22 09:15:27,618::__init__::225::IOProcess::(_processLogs) (190915) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:27,618::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:27,618::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:15:27,618::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:15:27,646::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000272214 s, 1.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) (190916) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) (190916) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:27,647::__init__::225::IOProcess::(_processLogs) (190916) Building response >Thread-14::DEBUG::2014-08-22 09:15:27,648::__init__::225::IOProcess::(_processLogs) (190916) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:27,648::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:15:27,648::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:15:27,850::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:15:27,850::task::595::Storage.TaskManager.Task::(_updateState) Task=`b40b87cc-fac0-4381-a470-7a8a92ad8640`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:27,850::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:15:27,850::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:27,851::task::1191::Storage.TaskManager.Task::(prepare) Task=`b40b87cc-fac0-4381-a470-7a8a92ad8640`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:27,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`b40b87cc-fac0-4381-a470-7a8a92ad8640`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:27,851::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:27,851::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:27,851::task::993::Storage.TaskManager.Task::(_decref) Task=`b40b87cc-fac0-4381-a470-7a8a92ad8640`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:15:27,882::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:15:27,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c2ec405-bbc1-4414-b44d-0272649b42cc`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:15:27,883::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:15:27,883::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`01e7e1ba-6bde-49ff-a0bc-814ea23e2b7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:15:27,883::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:15:27,883::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:15:27,883::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`01e7e1ba-6bde-49ff-a0bc-814ea23e2b7a`::Granted request >Thread-171::DEBUG::2014-08-22 09:15:27,884::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2c2ec405-bbc1-4414-b44d-0272649b42cc`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:15:27,884::task::993::Storage.TaskManager.Task::(_decref) Task=`2c2ec405-bbc1-4414-b44d-0272649b42cc`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:15:27,886::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:27,886::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c2ec405-bbc1-4414-b44d-0272649b42cc`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:27,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c2ec405-bbc1-4414-b44d-0272649b42cc`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:15:27,886::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:15:27,886::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:15:27,886::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:15:27,886::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:15:27,886::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:15:27,886::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:15:27,887::task::993::Storage.TaskManager.Task::(_decref) Task=`2c2ec405-bbc1-4414-b44d-0272649b42cc`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:29,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:29,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122185 s, 83.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:31,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:31,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116737 s, 87.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:31,823::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:31,823::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:31,823::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:31,823::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) (190917) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) (190917) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) (190917) Building response >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) (190917) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:31,824::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:15:31,825::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:15:31,853::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000282134 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:31,854::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:31,854::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:31,854::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:31,854::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:31,854::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) (190918) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) (190918) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) (190918) Building response >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) (190918) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:31,855::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:15:31,856::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:31,856::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:15:31,856::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:31,856::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:31,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) (190919) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) (190919) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) (190919) Building response >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) (190919) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:31,857::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) (190920) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) (190920) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) (190920) Building response >Thread-14::DEBUG::2014-08-22 09:15:31,858::__init__::225::IOProcess::(_processLogs) (190920) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:31,859::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) (190921) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) (190921) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) (190921) Building response >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) (190921) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:31,860::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:32,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:32,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:32,034::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:33,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:33,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119135 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:35,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:35,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116823 s, 87.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:15:35,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`39f60619-76bc-485e-9f62-9e6fa45fb256`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:35,802::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:15:35,803::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000272214', 'lastCheck': '8.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000282134', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:35,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`39f60619-76bc-485e-9f62-9e6fa45fb256`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000272214', 'lastCheck': '8.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000282134', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:35,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`39f60619-76bc-485e-9f62-9e6fa45fb256`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:35,803::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:35,803::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:35,803::task::993::Storage.TaskManager.Task::(_decref) Task=`39f60619-76bc-485e-9f62-9e6fa45fb256`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:15:35,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:37,039::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:37,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:37,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:37,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:37,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115864 s, 88.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:37,649::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:37,649::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:37,649::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:37,649::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:37,649::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) (190922) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) (190922) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190922) Building response >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) (190922) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:37,650::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:15:37,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:15:37,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000233893 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) (190923) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) (190923) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:37,680::__init__::225::IOProcess::(_processLogs) (190923) Building response >Thread-14::DEBUG::2014-08-22 09:15:37,681::__init__::225::IOProcess::(_processLogs) (190923) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:37,681::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:37,681::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:15:37,936::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:15:37,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`8322b753-2a0f-46f5-8362-963f73b55dbe`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:37,936::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:15:37,937::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:37,937::task::1191::Storage.TaskManager.Task::(prepare) Task=`8322b753-2a0f-46f5-8362-963f73b55dbe`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:37,937::task::595::Storage.TaskManager.Task::(_updateState) Task=`8322b753-2a0f-46f5-8362-963f73b55dbe`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:37,937::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:37,937::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:37,937::task::993::Storage.TaskManager.Task::(_decref) Task=`8322b753-2a0f-46f5-8362-963f73b55dbe`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:15:38,004::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:15:38,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`da9de367-946b-41b9-9dbe-f08d4ce41ec5`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:15:38,005::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:15:38,005::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7fb22d5b-09ed-4218-af7f-e8d94bd3a27e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:15:38,005::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:15:38,005::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:15:38,006::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7fb22d5b-09ed-4218-af7f-e8d94bd3a27e`::Granted request >Thread-171::DEBUG::2014-08-22 09:15:38,006::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`da9de367-946b-41b9-9dbe-f08d4ce41ec5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:15:38,006::task::993::Storage.TaskManager.Task::(_decref) Task=`da9de367-946b-41b9-9dbe-f08d4ce41ec5`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:15:38,008::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:38,008::task::1191::Storage.TaskManager.Task::(prepare) Task=`da9de367-946b-41b9-9dbe-f08d4ce41ec5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:38,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`da9de367-946b-41b9-9dbe-f08d4ce41ec5`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:15:38,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:15:38,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:15:38,008::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:15:38,009::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:15:38,009::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:15:38,009::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:15:38,009::task::993::Storage.TaskManager.Task::(_decref) Task=`da9de367-946b-41b9-9dbe-f08d4ce41ec5`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:39,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:39,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120954 s, 84.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:41,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:41,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117489 s, 87.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:41,861::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:41,861::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:41,861::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) (190924) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) (190924) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) (190924) Building response >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) (190924) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:41,862::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:15:41,863::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:15:41,891::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000236582 s, 2.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:41,892::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:41,892::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:41,892::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:41,892::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:41,892::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) (190925) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) (190925) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) (190925) Building response >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) (190925) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:41,893::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:15:41,894::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:41,894::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:15:41,894::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:41,894::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:41,894::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:41,894::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) (190926) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) (190926) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) (190926) Building response >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) (190926) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:41,895::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) (190927) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) (190927) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) (190927) Building response >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) (190927) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:41,896::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:41,897::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) (190928) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) (190928) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) (190928) Building response >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) (190928) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:41,898::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:42,044::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:42,045::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:42,045::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:43,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:43,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120142 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:45,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:45,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011887 s, 86.1 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:47,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:47,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:47,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:47,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-22 09:15:47,703::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:47,703::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:47,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:47,703::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:47,704::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:47,704::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:47,704::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:47,704::__init__::225::IOProcess::(_processLogs) (190929) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:47,704::__init__::225::IOProcess::(_processLogs) (190929) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:47,711::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190929) Building response >Thread-14::DEBUG::2014-08-22 09:15:47,712::__init__::225::IOProcess::(_processLogs) (190929) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:47,713::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:15:47,714::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:15:47,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:15:47,746::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227986 s, 1.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:47,748::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:47,748::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:47,749::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:47,749::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:47,751::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:47,751::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:47,752::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:47,752::__init__::225::IOProcess::(_processLogs) (190930) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:47,753::__init__::225::IOProcess::(_processLogs) (190930) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:47,753::__init__::225::IOProcess::(_processLogs) (190930) Building response >Thread-14::DEBUG::2014-08-22 09:15:47,754::__init__::225::IOProcess::(_processLogs) (190930) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:47,754::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:47,754::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:15:47,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0524298 s, 19.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:15:48,063::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:15:48,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`67f9e97b-32b3-4b19-a047-accaee56f8b3`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:48,064::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:15:48,064::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:48,064::task::1191::Storage.TaskManager.Task::(prepare) Task=`67f9e97b-32b3-4b19-a047-accaee56f8b3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:48,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`67f9e97b-32b3-4b19-a047-accaee56f8b3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:48,064::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:48,064::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:48,064::task::993::Storage.TaskManager.Task::(_decref) Task=`67f9e97b-32b3-4b19-a047-accaee56f8b3`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:15:48,125::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:15:48,125::task::595::Storage.TaskManager.Task::(_updateState) Task=`dfc1565f-8a8b-4849-9189-0194bbd4dd8a`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:15:48,125::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:15:48,126::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4c8525ca-bc1d-4161-88c5-b515c57a8792`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:15:48,126::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:15:48,126::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:15:48,126::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4c8525ca-bc1d-4161-88c5-b515c57a8792`::Granted request >Thread-171::DEBUG::2014-08-22 09:15:48,127::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dfc1565f-8a8b-4849-9189-0194bbd4dd8a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:15:48,128::task::993::Storage.TaskManager.Task::(_decref) Task=`dfc1565f-8a8b-4849-9189-0194bbd4dd8a`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:15:48,129::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:48,130::task::1191::Storage.TaskManager.Task::(prepare) Task=`dfc1565f-8a8b-4849-9189-0194bbd4dd8a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:48,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`dfc1565f-8a8b-4849-9189-0194bbd4dd8a`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:15:48,130::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:15:48,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:15:48,130::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:15:48,130::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:15:48,130::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:15:48,130::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:15:48,130::task::993::Storage.TaskManager.Task::(_decref) Task=`dfc1565f-8a8b-4849-9189-0194bbd4dd8a`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:49,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:49,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018419 s, 55.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:15:51,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`d02fd158-fdbb-46bb-90d9-f90d5f8b5ca2`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:51,016::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:15:51,016::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227986', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000236582', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:51,016::task::1191::Storage.TaskManager.Task::(prepare) Task=`d02fd158-fdbb-46bb-90d9-f90d5f8b5ca2`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227986', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000236582', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:15:51,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`d02fd158-fdbb-46bb-90d9-f90d5f8b5ca2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:51,017::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:51,017::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:51,017::task::993::Storage.TaskManager.Task::(_decref) Task=`d02fd158-fdbb-46bb-90d9-f90d5f8b5ca2`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:15:51,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:15:51,899::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:51,899::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:51,899::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:51,899::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) (190931) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) (190931) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) (190931) Building response >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) (190931) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-14::DEBUG::2014-08-22 09:15:51,900::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:15:51,901::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:51,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-22 09:15:51,957::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000195009 s, 2.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) (190932) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:51,958::__init__::225::IOProcess::(_processLogs) (190932) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:51,966::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190932) Building response >Thread-14::DEBUG::2014-08-22 09:15:51,967::__init__::225::IOProcess::(_processLogs) (190932) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:51,968::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:51,969::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:15:51,972::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:51,973::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:15:51,974::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:51,975::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:51,975::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:51,976::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:51,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:51,977::__init__::225::IOProcess::(_processLogs) (190933) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:51,978::__init__::225::IOProcess::(_processLogs) (190933) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:51,978::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190933) Building response >Thread-14::DEBUG::2014-08-22 09:15:51,979::__init__::225::IOProcess::(_processLogs) (190933) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:51,979::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:51,980::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:15:51,982::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:52,007::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:15:52,007::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:52,007::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:52,007::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:52,007::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) (190934) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) (190934) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190934) Building response >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) (190934) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:52,008::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:15:52,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247415 s, 41.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:52,010::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:52,011::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:15:52,011::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:52,012::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:52,012::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:52,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:52,014::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:52,014::__init__::225::IOProcess::(_processLogs) (190935) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:15:52,015::__init__::225::IOProcess::(_processLogs) (190935) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:15:52,015::__init__::225::IOProcess::(_processLogs) (190935) Building response >Thread-14::DEBUG::2014-08-22 09:15:52,016::__init__::225::IOProcess::(_processLogs) (190935) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:52,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:52,017::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:52,057::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:52,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:52,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:15:54,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:54,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118797 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:15:56,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:15:56,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114669 s, 89.3 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:57,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:57,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:15:57,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:57,783::__init__::225::IOProcess::(_processLogs) (190936) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:57,784::__init__::225::IOProcess::(_processLogs) (190936) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:57,784::__init__::225::IOProcess::(_processLogs) (190936) Building response >Thread-14::DEBUG::2014-08-22 09:15:57,784::__init__::225::IOProcess::(_processLogs) (190936) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:57,784::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:57,784::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:15:57,784::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:15:57,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000250734 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:15:57,813::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:15:57,813::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:15:57,813::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) (190937) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) (190937) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) (190937) Building response >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) (190937) Queuing response >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:15:57,814::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:15:58,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-13::DEBUG::2014-08-22 09:15:58,201::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:15:58,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ae8c3f3-4743-4953-9de6-af6c2beec663`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:15:58,201::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:15:58,202::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:58,202::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ae8c3f3-4743-4953-9de6-af6c2beec663`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:15:58,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ae8c3f3-4743-4953-9de6-af6c2beec663`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:15:58,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:15:58,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:15:58,202::task::993::Storage.TaskManager.Task::(_decref) Task=`7ae8c3f3-4743-4953-9de6-af6c2beec663`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:15:58,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116893 s, 87.6 MB/s\n'; <rc> = 0 >Thread-171::DEBUG::2014-08-22 09:15:58,253::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:15:58,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`958de405-383f-403d-9ae5-aaa24ba5ced2`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:15:58,254::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:15:58,254::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`304d058d-3b57-49b8-bbcc-e5f1b5089869`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:15:58,254::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:15:58,254::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:15:58,254::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`304d058d-3b57-49b8-bbcc-e5f1b5089869`::Granted request >Thread-171::DEBUG::2014-08-22 09:15:58,255::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`958de405-383f-403d-9ae5-aaa24ba5ced2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:15:58,255::task::993::Storage.TaskManager.Task::(_decref) Task=`958de405-383f-403d-9ae5-aaa24ba5ced2`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:15:58,257::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:58,257::task::1191::Storage.TaskManager.Task::(prepare) Task=`958de405-383f-403d-9ae5-aaa24ba5ced2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:15:58,257::task::595::Storage.TaskManager.Task::(_updateState) Task=`958de405-383f-403d-9ae5-aaa24ba5ced2`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:15:58,257::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:15:58,257::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:15:58,257::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:15:58,257::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:15:58,258::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:15:58,258::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:15:58,258::task::993::Storage.TaskManager.Task::(_decref) Task=`958de405-383f-403d-9ae5-aaa24ba5ced2`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:00,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:00,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117872 s, 86.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:02,020::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:02,021::__init__::225::IOProcess::(_processLogs) (190938) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:02,021::__init__::225::IOProcess::(_processLogs) (190938) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:02,021::__init__::225::IOProcess::(_processLogs) (190938) Building response >Thread-14::DEBUG::2014-08-22 09:16:02,021::__init__::225::IOProcess::(_processLogs) (190938) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:02,021::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:02,021::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:16:02,022::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:16:02,050::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000278096 s, 2.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:02,051::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:02,052::__init__::225::IOProcess::(_processLogs) (190939) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:02,052::__init__::225::IOProcess::(_processLogs) (190939) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:02,052::__init__::225::IOProcess::(_processLogs) (190939) Building response >Thread-14::DEBUG::2014-08-22 09:16:02,052::__init__::225::IOProcess::(_processLogs) (190939) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:02,052::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:02,052::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:02,053::__init__::225::IOProcess::(_processLogs) (190940) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) (190940) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) (190940) Building response >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) (190940) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:02,054::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) (190941) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) (190941) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) (190941) Building response >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) (190941) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:02,055::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:02,056::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:02,057::__init__::225::IOProcess::(_processLogs) (190942) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:02,057::__init__::225::IOProcess::(_processLogs) (190942) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:02,057::__init__::225::IOProcess::(_processLogs) (190942) Building response >Thread-14::DEBUG::2014-08-22 09:16:02,057::__init__::225::IOProcess::(_processLogs) (190942) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:02,057::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:02,057::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:02,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:02,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:02,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:16:02,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:02,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120393 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:04,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:04,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115593 s, 88.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:16:06,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`c35a7160-e279-43ef-b9b1-84e042f04a4e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:06,260::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:16:06,260::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000250734', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000278096', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:06,261::task::1191::Storage.TaskManager.Task::(prepare) Task=`c35a7160-e279-43ef-b9b1-84e042f04a4e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000250734', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000278096', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:06,261::task::595::Storage.TaskManager.Task::(_updateState) Task=`c35a7160-e279-43ef-b9b1-84e042f04a4e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:06,261::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:06,261::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:06,261::task::993::Storage.TaskManager.Task::(_decref) Task=`c35a7160-e279-43ef-b9b1-84e042f04a4e`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:16:06,286::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:16:06,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:06,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118767 s, 86.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:07,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:07,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:07,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) (190943) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) (190943) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:07,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190943) Building response >Thread-14::DEBUG::2014-08-22 09:16:07,817::__init__::225::IOProcess::(_processLogs) (190943) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:07,817::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:07,817::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:16:07,817::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:16:07,845::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000249842 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:07,846::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:07,846::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:07,846::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:07,846::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) (190944) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) (190944) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) (190944) Building response >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) (190944) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:07,847::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:16:08,323::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:16:08,323::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f2a2f77-616c-485a-9ede-0a5648e5448b`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:08,323::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:16:08,323::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:08,324::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f2a2f77-616c-485a-9ede-0a5648e5448b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:08,324::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f2a2f77-616c-485a-9ede-0a5648e5448b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:08,324::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:08,324::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:08,324::task::993::Storage.TaskManager.Task::(_decref) Task=`2f2a2f77-616c-485a-9ede-0a5648e5448b`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:16:08,347::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:16:08,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e25b23e-76bd-4736-8065-a659e35ed2d4`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:16:08,348::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:16:08,348::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8b71b0c0-20ca-4f6e-bcec-ae4be66b1b00`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:16:08,348::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:16:08,348::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:16:08,348::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8b71b0c0-20ca-4f6e-bcec-ae4be66b1b00`::Granted request >Thread-171::DEBUG::2014-08-22 09:16:08,349::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0e25b23e-76bd-4736-8065-a659e35ed2d4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:16:08,349::task::993::Storage.TaskManager.Task::(_decref) Task=`0e25b23e-76bd-4736-8065-a659e35ed2d4`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:16:08,351::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:08,351::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e25b23e-76bd-4736-8065-a659e35ed2d4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:08,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e25b23e-76bd-4736-8065-a659e35ed2d4`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:16:08,351::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:16:08,351::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:16:08,351::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:16:08,351::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:16:08,351::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:16:08,352::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:16:08,352::task::993::Storage.TaskManager.Task::(_decref) Task=`0e25b23e-76bd-4736-8065-a659e35ed2d4`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:08,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:08,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118535 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:10,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:10,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118839 s, 86.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:12,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:12,058::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:12,058::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:12,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) (190945) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) (190945) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) (190945) Building response >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) (190945) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:12,059::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:16:12,060::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:12,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-22 09:16:12,089::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000160781 s, 3.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:12,091::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:12,092::__init__::225::IOProcess::(_processLogs) Message size is 165 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:12,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:12,092::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:12,093::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:12,094::__init__::225::IOProcess::(_processLogs) Waiting for next request... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:12,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:12,094::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:12,095::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:12,095::__init__::225::IOProcess::(_processLogs) (190946) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:12,096::__init__::225::IOProcess::(_processLogs) (190946) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:12,096::__init__::225::IOProcess::(_processLogs) (190946) Building response >Thread-14::DEBUG::2014-08-22 09:16:12,096::__init__::225::IOProcess::(_processLogs) (190946) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:12,096::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:12,096::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) (190947) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) (190947) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:12,097::__init__::225::IOProcess::(_processLogs) (190947) Building response >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) (190947) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:12,098::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) (190948) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) (190948) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) (190948) Building response >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) (190948) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:12,099::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) (190949) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:12,100::__init__::225::IOProcess::(_processLogs) (190949) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:12,101::__init__::225::IOProcess::(_processLogs) (190949) Building response >Thread-14::DEBUG::2014-08-22 09:16:12,101::__init__::225::IOProcess::(_processLogs) (190949) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:12,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:12,101::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:16:12,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:12,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116941 s, 87.6 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:16:13,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d593117-c5c4-42ed-843d-e792d264aa40`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:16:13,762::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:16:13,762::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:13,762::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) (190950) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) (190950) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(190950) Building response >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) (190950) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:13,763::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:16:13,764::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:13,764::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) (190951) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) (190951) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) (190951) Building response >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) (190951) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:13,765::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:16:13,766::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:16:13,766::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d593117-c5c4-42ed-843d-e792d264aa40`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:16:13,766::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d593117-c5c4-42ed-843d-e792d264aa40`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:16:13,766::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:16:13,766::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:16:13,766::task::993::Storage.TaskManager.Task::(_decref) Task=`4d593117-c5c4-42ed-843d-e792d264aa40`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:14,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:14,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018627 s, 55.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:16,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:16,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115132 s, 88.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:17,101::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:17,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:17,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:17,848::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) (190952) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) (190952) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) (190952) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:17,849::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:17,850::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:16:17,850::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:16:17,878::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000249108 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:17,879::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:17,879::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:17,879::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:17,879::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) (190953) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) (190953) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) (190953) Building response >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) (190953) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:17,880::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:16:18,392::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [5a785a8f] >Thread-13::DEBUG::2014-08-22 09:16:18,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`adea61de-e687-46cf-bb6e-d1dc888f9a6d`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:18,393::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:16:18,393::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:18,393::task::1191::Storage.TaskManager.Task::(prepare) Task=`adea61de-e687-46cf-bb6e-d1dc888f9a6d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:18,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`adea61de-e687-46cf-bb6e-d1dc888f9a6d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:18,393::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:18,393::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:18,393::task::993::Storage.TaskManager.Task::(_decref) Task=`adea61de-e687-46cf-bb6e-d1dc888f9a6d`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:16:18,419::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [5a785a8f] >Thread-171::DEBUG::2014-08-22 09:16:18,419::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebc25991-02d1-4e44-9894-6b51f24f35e9`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:16:18,419::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:16:18,419::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5aa29806-099b-4dd6-b468-e031cdd2cd4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:16:18,420::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:16:18,420::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:16:18,420::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5aa29806-099b-4dd6-b468-e031cdd2cd4a`::Granted request >Thread-171::DEBUG::2014-08-22 09:16:18,420::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ebc25991-02d1-4e44-9894-6b51f24f35e9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:16:18,420::task::993::Storage.TaskManager.Task::(_decref) Task=`ebc25991-02d1-4e44-9894-6b51f24f35e9`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:16:18,422::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:18,422::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebc25991-02d1-4e44-9894-6b51f24f35e9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:18,422::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebc25991-02d1-4e44-9894-6b51f24f35e9`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:16:18,423::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:16:18,423::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:16:18,423::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:16:18,423::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:16:18,423::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:16:18,423::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:16:18,423::task::993::Storage.TaskManager.Task::(_decref) Task=`ebc25991-02d1-4e44-9894-6b51f24f35e9`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:18,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:18,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118254 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:20,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:21,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114038 s, 89.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:16:21,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e053cae-93cb-4adf-92af-0c7e377f9321`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:21,570::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:16:21,570::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000249108', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000160781', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:21,570::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e053cae-93cb-4adf-92af-0c7e377f9321`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000249108', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000160781', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:21,570::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e053cae-93cb-4adf-92af-0c7e377f9321`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:21,570::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:21,570::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:21,570::task::993::Storage.TaskManager.Task::(_decref) Task=`0e053cae-93cb-4adf-92af-0c7e377f9321`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:16:21,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) (190954) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:22,102::__init__::225::IOProcess::(_processLogs) (190954) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:22,103::__init__::225::IOProcess::(_processLogs) (190954) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:22,103::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:22,103::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:16:22,103::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:22,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-22 09:16:22,133::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00020429 s, 2.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:22,135::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:22,135::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:22,136::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:22,136::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:22,137::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:22,137::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:22,138::__init__::225::IOProcess::(_processLogs) (190955) Finding callback 'statvfs'... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:22,138::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:22,138::__init__::225::IOProcess::(_processLogs) (190955) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:22,139::__init__::225::IOProcess::(_processLogs) (190955) Building response >Thread-14::DEBUG::2014-08-22 09:16:22,139::__init__::225::IOProcess::(_processLogs) (190955) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:22,140::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:22,140::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) (190956) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) (190956) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:22,141::__init__::225::IOProcess::(_processLogs) (190956) Building response >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) (190956) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:22,142::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) (190957) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) (190957) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) (190957) Building response >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) (190957) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:22,143::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) (190958) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) (190958) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) (190958) Building response >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) (190958) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:22,144::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:16:23,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:23,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118526 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:25,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:25,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117129 s, 87.4 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:27,145::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:27,146::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:27,146::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:16:27,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:27,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011611 s, 88.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:27,881::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:27,881::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) (190959) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) (190959) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) (190959) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:27,882::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:16:27,883::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:16:27,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000289719 s, 1.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) (190960) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) (190960) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) (190960) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:27,912::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:16:28,473::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:16:28,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`7804f7dd-52c5-4978-8477-b5563d229d61`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:28,475::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:16:28,476::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:28,478::task::1191::Storage.TaskManager.Task::(prepare) Task=`7804f7dd-52c5-4978-8477-b5563d229d61`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:28,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`7804f7dd-52c5-4978-8477-b5563d229d61`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:28,496::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:28,497::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:28,498::task::993::Storage.TaskManager.Task::(_decref) Task=`7804f7dd-52c5-4978-8477-b5563d229d61`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:16:28,535::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:16:28,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`73b89fa5-0d7d-4aee-bb7c-d3a442edadfb`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:16:28,536::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:16:28,536::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e8be435f-2b1d-4a96-999e-9304b1ea9b70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:16:28,536::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:16:28,537::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:16:28,537::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e8be435f-2b1d-4a96-999e-9304b1ea9b70`::Granted request >Thread-171::DEBUG::2014-08-22 09:16:28,537::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`73b89fa5-0d7d-4aee-bb7c-d3a442edadfb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:16:28,537::task::993::Storage.TaskManager.Task::(_decref) Task=`73b89fa5-0d7d-4aee-bb7c-d3a442edadfb`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:16:28,541::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:28,541::task::1191::Storage.TaskManager.Task::(prepare) Task=`73b89fa5-0d7d-4aee-bb7c-d3a442edadfb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:28,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`73b89fa5-0d7d-4aee-bb7c-d3a442edadfb`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:16:28,541::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:16:28,541::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:16:28,542::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:16:28,542::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:16:28,542::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:16:28,542::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:16:28,542::task::993::Storage.TaskManager.Task::(_decref) Task=`73b89fa5-0d7d-4aee-bb7c-d3a442edadfb`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:29,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:29,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122776 s, 83.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:31,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:31,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141951 s, 72.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) (190961) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) (190961) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) (190961) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:32,146::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:16:32,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:32,176::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-22 09:16:32,176::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000182601 s, 3.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:32,179::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:32,180::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:32,180::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:32,180::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:32,182::__init__::225::IOProcess::(_processLogs) Waiting for next request... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:32,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:32,183::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:32,183::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:32,184::__init__::225::IOProcess::(_processLogs) (190962) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:32,185::__init__::225::IOProcess::(_processLogs) (190962) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:32,185::__init__::225::IOProcess::(_processLogs) (190962) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:32,185::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:32,185::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) (190963) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) (190963) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) (190963) Building response >Thread-14::DEBUG::2014-08-22 09:16:32,186::__init__::225::IOProcess::(_processLogs) (190963) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:32,187::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:32,187::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:32,187::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:16:32,187::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:32,187::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:32,187::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) (190964) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) (190964) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) (190964) Building response >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) (190964) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:32,188::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) (190965) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) (190965) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) (190965) Building response >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) (190965) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:32,189::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:16:33,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:33,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118553 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:35,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:35,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118471 s, 86.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:16:36,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2e36517-a2e3-4fbc-b598-f63a9c1b4210`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:36,787::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:16:36,787::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000289719', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182601', 'lastCheck': '4.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:36,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2e36517-a2e3-4fbc-b598-f63a9c1b4210`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000289719', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182601', 'lastCheck': '4.6', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:36,787::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2e36517-a2e3-4fbc-b598-f63a9c1b4210`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:36,787::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:36,787::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:36,787::task::993::Storage.TaskManager.Task::(_decref) Task=`c2e36517-a2e3-4fbc-b598-f63a9c1b4210`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:16:36,812::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:37,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:37,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:37,191::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Dummy-170::DEBUG::2014-08-22 09:16:37,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:37,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116162 s, 88.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) (190966) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:37,914::__init__::225::IOProcess::(_processLogs) (190966) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:37,915::__init__::225::IOProcess::(_processLogs) (190966) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:37,915::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:37,915::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:16:37,915::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:16:37,943::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000175353 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) (190967) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) (190967) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) (190967) Building response >Thread-14::DEBUG::2014-08-22 09:16:37,944::__init__::225::IOProcess::(_processLogs) (190967) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:37,945::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:37,945::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:16:38,593::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:16:38,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`a76c3596-ac06-4f80-9ed9-08c722d42122`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:38,593::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:16:38,593::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:38,593::task::1191::Storage.TaskManager.Task::(prepare) Task=`a76c3596-ac06-4f80-9ed9-08c722d42122`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:38,594::task::595::Storage.TaskManager.Task::(_updateState) Task=`a76c3596-ac06-4f80-9ed9-08c722d42122`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:38,594::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:38,594::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:38,594::task::993::Storage.TaskManager.Task::(_decref) Task=`a76c3596-ac06-4f80-9ed9-08c722d42122`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:16:38,618::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:16:38,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba647508-5e63-4cf9-bd2f-27e3adbe5e8d`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:16:38,618::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:16:38,619::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9af6b0d1-2558-4a96-b89d-04bcae3b0cea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:16:38,619::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:16:38,619::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:16:38,619::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9af6b0d1-2558-4a96-b89d-04bcae3b0cea`::Granted request >Thread-171::DEBUG::2014-08-22 09:16:38,619::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ba647508-5e63-4cf9-bd2f-27e3adbe5e8d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:16:38,619::task::993::Storage.TaskManager.Task::(_decref) Task=`ba647508-5e63-4cf9-bd2f-27e3adbe5e8d`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:16:38,621::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:38,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba647508-5e63-4cf9-bd2f-27e3adbe5e8d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:38,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba647508-5e63-4cf9-bd2f-27e3adbe5e8d`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:16:38,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:16:38,622::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:16:38,622::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:16:38,622::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:16:38,622::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:16:38,622::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:16:38,622::task::993::Storage.TaskManager.Task::(_decref) Task=`ba647508-5e63-4cf9-bd2f-27e3adbe5e8d`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:39,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:39,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204934 s, 50.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:41,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:41,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124723 s, 82.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:42,190::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) (190968) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) (190968) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) (190968) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:42,191::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:16:42,192::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:42,221::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-22 09:16:42,222::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000902291 s, 632 kB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:42,222::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:42,224::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:42,225::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:42,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:42,226::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:42,227::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:42,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:42,228::__init__::225::IOProcess::(_processLogs) (190969) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:42,228::__init__::225::IOProcess::(_processLogs) (190969) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:42,228::__init__::225::IOProcess::(_processLogs) (190969) Building response >Thread-14::DEBUG::2014-08-22 09:16:42,229::__init__::225::IOProcess::(_processLogs) (190969) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:42,229::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:42,229::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:16:42,229::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) (190970) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) (190970) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) (190970) Building response >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) (190970) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:42,230::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) (190971) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) (190971) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:42,231::__init__::225::IOProcess::(_processLogs) (190971) Building response >Thread-14::DEBUG::2014-08-22 09:16:42,232::__init__::225::IOProcess::(_processLogs) (190971) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:42,232::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:42,232::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:42,232::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:16:42,232::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:42,232::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) (190972) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) (190972) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) (190972) Building response >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) (190972) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:42,233::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:16:43,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:43,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118112 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:45,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:46,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116655 s, 87.8 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:47,234::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:47,234::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:47,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) (190973) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:47,946::__init__::225::IOProcess::(_processLogs) (190973) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:47,947::__init__::225::IOProcess::(_processLogs) (190973) Building response >Thread-14::DEBUG::2014-08-22 09:16:47,947::__init__::225::IOProcess::(_processLogs) (190973) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:47,947::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:16:47,947::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:16:47,975::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000243423 s, 1.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:47,976::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:47,976::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:47,976::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:47,976::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:47,976::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:47,977::__init__::225::IOProcess::(_processLogs) (190974) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:47,977::__init__::225::IOProcess::(_processLogs) (190974) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:47,977::__init__::225::IOProcess::(_processLogs) (190974) Building response >Thread-14::DEBUG::2014-08-22 09:16:47,977::__init__::225::IOProcess::(_processLogs) (190974) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:47,977::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:47,977::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:16:48,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:48,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118304 s, 86.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:16:48,671::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:16:48,671::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d8b5d2c-c55a-4e05-aca6-4bb3ce676e70`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:48,671::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:16:48,672::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:48,672::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d8b5d2c-c55a-4e05-aca6-4bb3ce676e70`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:48,672::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d8b5d2c-c55a-4e05-aca6-4bb3ce676e70`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:48,672::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:48,672::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:48,672::task::993::Storage.TaskManager.Task::(_decref) Task=`0d8b5d2c-c55a-4e05-aca6-4bb3ce676e70`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:16:48,706::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:16:48,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d94bac8-ac38-4501-b06b-402c3344ecdb`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:16:48,706::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:16:48,707::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c53624ea-910d-40a3-867c-65d5e578f52e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:16:48,707::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:16:48,707::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:16:48,707::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c53624ea-910d-40a3-867c-65d5e578f52e`::Granted request >Thread-171::DEBUG::2014-08-22 09:16:48,707::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0d94bac8-ac38-4501-b06b-402c3344ecdb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:16:48,707::task::993::Storage.TaskManager.Task::(_decref) Task=`0d94bac8-ac38-4501-b06b-402c3344ecdb`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:16:48,709::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:48,710::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d94bac8-ac38-4501-b06b-402c3344ecdb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:48,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d94bac8-ac38-4501-b06b-402c3344ecdb`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:16:48,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:16:48,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:16:48,710::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:16:48,710::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:16:48,710::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:16:48,710::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:16:48,710::task::993::Storage.TaskManager.Task::(_decref) Task=`0d94bac8-ac38-4501-b06b-402c3344ecdb`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:16:50,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:50,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179879 s, 56.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:16:52,025::task::595::Storage.TaskManager.Task::(_updateState) Task=`0cab63cb-0a63-4c0e-9787-00a3a8e03767`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:52,025::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:16:52,025::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000243423', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000902291', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:52,025::task::1191::Storage.TaskManager.Task::(prepare) Task=`0cab63cb-0a63-4c0e-9787-00a3a8e03767`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000243423', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000902291', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:16:52,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`0cab63cb-0a63-4c0e-9787-00a3a8e03767`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:52,026::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:52,026::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:52,026::task::993::Storage.TaskManager.Task::(_decref) Task=`0cab63cb-0a63-4c0e-9787-00a3a8e03767`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:16:52,053::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:52,234::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:52,234::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:52,234::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:52,234::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:52,235::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:52,235::__init__::225::IOProcess::(_processLogs) (190975) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:52,235::__init__::225::IOProcess::(_processLogs) (190975) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:52,235::__init__::225::IOProcess::(_processLogs) (190975) Building response >Thread-14::DEBUG::2014-08-22 09:16:52,235::__init__::225::IOProcess::(_processLogs) (190975) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:52,235::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:16:52,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:52,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:52,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-22 09:16:52,266::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000258197 s, 2.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:52,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:52,298::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:16:52,299::__init__::225::IOProcess::(_processLogs) Marshaling message... >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:52,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:52,300::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:52,300::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:52,301::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:52,302::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:52,302::__init__::225::IOProcess::(_processLogs) (190976) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:52,302::__init__::225::IOProcess::(_processLogs) (190976) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:52,304::__init__::225::IOProcess::(_processLogs) (190976) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:52,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:52,308::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:16:52,309::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:52,310::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:52,311::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:52,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:52,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:52,313::__init__::225::IOProcess::(_processLogs) (190977) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:52,313::__init__::225::IOProcess::(_processLogs) (190977) Building response >Thread-14::DEBUG::2014-08-22 09:16:52,314::__init__::225::IOProcess::(_processLogs) (190977) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:52,315::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:52,315::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:52,317::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:16:52,318::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:52,318::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:52,344::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Dummy-170::DEBUG::2014-08-22 09:16:52,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234185 s, 43.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:52,345::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:52,346::__init__::225::IOProcess::(_processLogs) (190978) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:52,346::__init__::225::IOProcess::(_processLogs) (190978) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:52,346::__init__::225::IOProcess::(_processLogs) (190978) Building response >Thread-14::DEBUG::2014-08-22 09:16:52,347::__init__::225::IOProcess::(_processLogs) (190978) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:52,347::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:52,348::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:16:52,349::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:16:52,350::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:52,350::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:16:52,351::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:52,351::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:52,352::__init__::225::IOProcess::(_processLogs) (190979) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:16:52,352::__init__::225::IOProcess::(_processLogs) (190979) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:16:52,352::__init__::225::IOProcess::(_processLogs) (190979) Building response >Thread-14::DEBUG::2014-08-22 09:16:52,353::__init__::225::IOProcess::(_processLogs) (190979) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:52,353::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:52,353::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:16:54,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:54,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117739 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:16:56,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:56,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115187 s, 88.9 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:57,308::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:57,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:16:57,314::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:16:57,979::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:57,979::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:57,980::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:57,980::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:57,980::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:57,980::__init__::225::IOProcess::(_processLogs) (190980) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:57,980::__init__::225::IOProcess::(_processLogs) (190980) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:57,981::__init__::225::IOProcess::(_processLogs) (190980) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:57,981::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:16:57,981::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:16:58,015::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000295504 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:16:58,016::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:16:58,016::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:16:58,016::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:16:58,016::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:16:58,017::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:16:58,017::__init__::225::IOProcess::(_processLogs) (190981) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:16:58,017::__init__::225::IOProcess::(_processLogs) (190981) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:16:58,017::__init__::225::IOProcess::(_processLogs) (190981) Queuing response >Thread-14::DEBUG::2014-08-22 09:16:58,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:16:58,017::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:16:58,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:16:58,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119747 s, 85.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-08-22 09:16:58,801::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:16:58,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffe61e14-e997-4348-b6fc-1468ad2bc781`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:16:58,801::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:16:58,802::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:58,802::task::1191::Storage.TaskManager.Task::(prepare) Task=`ffe61e14-e997-4348-b6fc-1468ad2bc781`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:16:58,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffe61e14-e997-4348-b6fc-1468ad2bc781`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:16:58,802::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:16:58,802::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:16:58,802::task::993::Storage.TaskManager.Task::(_decref) Task=`ffe61e14-e997-4348-b6fc-1468ad2bc781`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:16:58,835::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:16:58,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`849dc362-8c10-4a5b-b672-2f5beb6a5c0f`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:16:58,835::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:16:58,835::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`46254def-a9d0-4138-ba47-d4287982195b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:16:58,836::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:16:58,836::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:16:58,836::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`46254def-a9d0-4138-ba47-d4287982195b`::Granted request >Thread-171::DEBUG::2014-08-22 09:16:58,836::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`849dc362-8c10-4a5b-b672-2f5beb6a5c0f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:16:58,836::task::993::Storage.TaskManager.Task::(_decref) Task=`849dc362-8c10-4a5b-b672-2f5beb6a5c0f`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:16:58,839::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:58,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`849dc362-8c10-4a5b-b672-2f5beb6a5c0f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:16:58,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`849dc362-8c10-4a5b-b672-2f5beb6a5c0f`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:16:58,839::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:16:58,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:16:58,840::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:16:58,840::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:16:58,840::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:16:58,840::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:16:58,840::task::993::Storage.TaskManager.Task::(_decref) Task=`849dc362-8c10-4a5b-b672-2f5beb6a5c0f`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:17:00,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:00,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156233 s, 65.5 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:02,320::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:02,320::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:02,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:17:02,355::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:17:02,355::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) (190982) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) (190982) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) (190982) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:02,356::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:17:02,357::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:17:02,385::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000275391 s, 2.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) (190983) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:02,386::__init__::225::IOProcess::(_processLogs) (190983) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:02,387::__init__::225::IOProcess::(_processLogs) (190983) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:02,387::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:02,387::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) (190984) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) (190984) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) (190984) Building response >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) (190984) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:02,388::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:02,389::__init__::225::IOProcess::(_processLogs) (190985) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) (190985) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) (190985) Building response >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) (190985) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:17:02,390::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) (190986) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) (190986) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) (190986) Building response >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) (190986) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:02,391::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:17:02,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:02,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114959 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:17:04,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:04,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116634 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:17:06,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:06,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116956 s, 87.6 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:07,327::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:07,327::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:07,327::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-13::DEBUG::2014-08-22 09:17:07,333::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f530fbb-9f57-420e-9c66-0bd0919f0861`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:17:07,333::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:17:07,334::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000295504', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000275391', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:17:07,334::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f530fbb-9f57-420e-9c66-0bd0919f0861`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000295504', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000275391', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:17:07,334::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f530fbb-9f57-420e-9c66-0bd0919f0861`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:17:07,334::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:17:07,334::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:17:07,334::task::993::Storage.TaskManager.Task::(_decref) Task=`3f530fbb-9f57-420e-9c66-0bd0919f0861`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:17:07,356::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:17:08,018::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:17:08,018::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:08,018::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) (190987) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) (190987) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) (190987) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:08,019::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:17:08,020::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:17:08,048::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000339548 s, 1.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) (190988) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) (190988) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) (190988) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:08,049::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Dummy-170::DEBUG::2014-08-22 09:17:08,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-13::DEBUG::2014-08-22 09:17:08,918::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:17:08,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba1bb0b5-7ab7-4fda-bfd2-243e3c9bd460`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:17:08,919::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:17:08,920::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:17:08,920::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba1bb0b5-7ab7-4fda-bfd2-243e3c9bd460`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:17:08,920::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba1bb0b5-7ab7-4fda-bfd2-243e3c9bd460`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:17:08,920::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:17:08,920::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:17:08,920::task::993::Storage.TaskManager.Task::(_decref) Task=`ba1bb0b5-7ab7-4fda-bfd2-243e3c9bd460`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:17:08,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120564 s, 84.9 MB/s\n'; <rc> = 0 >Thread-171::DEBUG::2014-08-22 09:17:08,980::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:17:08,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`e91d694e-3ba4-421e-b814-7570c74123a3`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:17:08,980::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:17:08,980::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`231c696a-bdda-4cdd-9cfc-d129959ddea4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:17:08,980::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:17:08,981::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:17:08,981::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`231c696a-bdda-4cdd-9cfc-d129959ddea4`::Granted request >Thread-171::DEBUG::2014-08-22 09:17:08,981::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e91d694e-3ba4-421e-b814-7570c74123a3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:17:08,981::task::993::Storage.TaskManager.Task::(_decref) Task=`e91d694e-3ba4-421e-b814-7570c74123a3`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:17:08,983::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:17:08,983::task::1191::Storage.TaskManager.Task::(prepare) Task=`e91d694e-3ba4-421e-b814-7570c74123a3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:17:08,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`e91d694e-3ba4-421e-b814-7570c74123a3`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:17:08,983::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:17:08,984::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:17:08,984::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:17:08,984::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:17:08,984::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:17:08,984::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:17:08,984::task::993::Storage.TaskManager.Task::(_decref) Task=`e91d694e-3ba4-421e-b814-7570c74123a3`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:17:10,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:11,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114776 s, 89.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:12,333::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:12,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:12,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) (190989) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) (190989) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:12,397::__init__::225::IOProcess::(_processLogs) (190989) Building response >Thread-14::DEBUG::2014-08-22 09:17:12,398::__init__::225::IOProcess::(_processLogs) (190989) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:12,398::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:12,398::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:17:12,398::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:17:12,426::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021267 s, 2.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:17:12,427::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:17:12,427::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:12,427::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:12,427::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:12,427::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:12,428::__init__::225::IOProcess::(_processLogs) (190990) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:12,428::__init__::225::IOProcess::(_processLogs) (190990) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:12,428::__init__::225::IOProcess::(_processLogs) (190990) Building response >Thread-14::DEBUG::2014-08-22 09:17:12,428::__init__::225::IOProcess::(_processLogs) (190990) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:12,428::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:12,428::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) (190991) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) (190991) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:12,429::__init__::225::IOProcess::(_processLogs) (190991) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:12,430::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) (190992) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) (190992) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) (190992) Building response >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) (190992) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:17:12,431::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) (190993) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) (190993) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) (190993) Building response >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) (190993) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:12,432::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Dummy-170::DEBUG::2014-08-22 09:17:13,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:13,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115518 s, 88.6 MB/s\n'; <rc> = 0 >Thread-183::DEBUG::2014-08-22 09:17:13,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`967a3658-3b30-482b-b24f-d71c24bfc834`::moving from state init -> state preparing >Thread-183::INFO::2014-08-22 09:17:13,831::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-14::DEBUG::2014-08-22 09:17:13,831::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:17:13,832::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:13,832::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:13,832::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:13,832::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:13,832::__init__::225::IOProcess::(_processLogs) (190994) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:17:13,832::__init__::225::IOProcess::(_processLogs) (190994) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:17:13,833::__init__::225::IOProcess::(_processLogs) (190994) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:13,833::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) Message size is 243 >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) (190995) Finding callback 'stat'... >Thread-14::DEBUG::2014-08-22 09:17:13,834::__init__::225::IOProcess::(_processLogs) (190995) Got request for method 'stat' >Thread-14::DEBUG::2014-08-22 09:17:13,835::__init__::225::IOProcess::(_processLogs) (190995) Building response >Thread-14::DEBUG::2014-08-22 09:17:13,835::__init__::225::IOProcess::(_processLogs) (190995) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:13,835::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:13,835::__init__::225::IOProcess::(_processLogs) Sending response sized 334 >Thread-183::INFO::2014-08-22 09:17:13,835::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:17:13,835::task::1191::Storage.TaskManager.Task::(prepare) Task=`967a3658-3b30-482b-b24f-d71c24bfc834`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-183::DEBUG::2014-08-22 09:17:13,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`967a3658-3b30-482b-b24f-d71c24bfc834`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-08-22 09:17:13,835::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-08-22 09:17:13,836::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-08-22 09:17:13,836::task::993::Storage.TaskManager.Task::(_decref) Task=`967a3658-3b30-482b-b24f-d71c24bfc834`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:17:15,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:15,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118137 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-170::DEBUG::2014-08-22 09:17:17,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:17,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116076 s, 88.2 MB/s\n'; <rc> = 0 >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:17,339::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:17,340::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >GuestMonitor-VM_for_allocation::DEBUG::2014-08-22 09:17:17,340::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) (190996) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) (190996) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) (190996) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:18,051::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-91::DEBUG::2014-08-22 09:17:18,052::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-91::DEBUG::2014-08-22 09:17:18,081::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000697801 s, 569 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:17:18,082::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-14::DEBUG::2014-08-22 09:17:18,082::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:18,082::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:18,082::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:18,082::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:18,082::__init__::225::IOProcess::(_processLogs) (190997) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:18,083::__init__::225::IOProcess::(_processLogs) (190997) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:18,083::__init__::225::IOProcess::(_processLogs) (190997) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:18,084::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-13::DEBUG::2014-08-22 09:17:19,034::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:17:19,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`3500442b-3e65-45cd-95bf-8e70f467aa0e`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:17:19,034::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:17:19,034::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:17:19,035::task::1191::Storage.TaskManager.Task::(prepare) Task=`3500442b-3e65-45cd-95bf-8e70f467aa0e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:17:19,035::task::595::Storage.TaskManager.Task::(_updateState) Task=`3500442b-3e65-45cd-95bf-8e70f467aa0e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:17:19,035::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:17:19,035::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:17:19,035::task::993::Storage.TaskManager.Task::(_decref) Task=`3500442b-3e65-45cd-95bf-8e70f467aa0e`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:17:19,128::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:17:19,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`926e35eb-3da0-4b98-98a3-3920da28c69e`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:17:19,129::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:17:19,129::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e4bfda37-26a8-46b5-b1c5-09d34cfcf716`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:17:19,129::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:17:19,129::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:17:19,129::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e4bfda37-26a8-46b5-b1c5-09d34cfcf716`::Granted request >Thread-171::DEBUG::2014-08-22 09:17:19,130::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`926e35eb-3da0-4b98-98a3-3920da28c69e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:17:19,130::task::993::Storage.TaskManager.Task::(_decref) Task=`926e35eb-3da0-4b98-98a3-3920da28c69e`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:17:19,132::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:17:19,132::task::1191::Storage.TaskManager.Task::(prepare) Task=`926e35eb-3da0-4b98-98a3-3920da28c69e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:17:19,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`926e35eb-3da0-4b98-98a3-3920da28c69e`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:17:19,132::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:17:19,132::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:17:19,132::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:17:19,132::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:17:19,132::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:17:19,133::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:17:19,133::task::993::Storage.TaskManager.Task::(_decref) Task=`926e35eb-3da0-4b98-98a3-3920da28c69e`::ref 0 aborting False >Dummy-170::DEBUG::2014-08-22 09:17:19,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-170::DEBUG::2014-08-22 09:17:19,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255501 s, 40.1 MB/s\n'; <rc> = 0 >MainThread::DEBUG::2014-08-22 09:17:20,706::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-08-22 09:17:20,710::protocoldetector::133::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-08-22 09:17:20,710::__init__::557::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-08-22 09:17:20,710::protocoldetector::104::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-08-22 09:17:20,711::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-08-22 09:17:20,711::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-08-22 09:17:20,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`8081b024-0e57-451c-ad5c-dd952c3f994e`::moving from state init -> state preparing >MainThread::INFO::2014-08-22 09:17:20,712::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-08-22 09:17:20,712::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >VM Channels Listener::INFO::2014-08-22 09:17:21,430::vmchannels::183::vds::(run) VM channels listener thread has ended. >Dummy-170::INFO::2014-08-22 09:17:21,720::storage_mailbox::792::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped >Thread-25::INFO::2014-08-22 09:17:22,145::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-25::INFO::2014-08-22 09:17:22,145::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-25::DEBUG::2014-08-22 09:17:22,145::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-25::DEBUG::2014-08-22 09:17:22,242::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.0692372 s, 59.2 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:17:22,433::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:17:22,433::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) (190998) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) (190998) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) (190998) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:22,434::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-16::DEBUG::2014-08-22 09:17:22,435::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-22 09:17:22,463::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000252658 s, 2.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-22 09:17:22,463::__init__::225::IOProcess::(_processLogs) Message size is 165 >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) (190999) Finding callback 'statvfs'... >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) (190999) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) (190999) Building response >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) (190999) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:22,464::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:22,465::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-14::DEBUG::2014-08-22 09:17:22,465::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-14::DEBUG::2014-08-22 09:17:22,465::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) (191000) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) (191000) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) (191000) Building response >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) (191000) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:22,466::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) (191001) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) (191001) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) (191001) Building response >Thread-14::DEBUG::2014-08-22 09:17:22,467::__init__::225::IOProcess::(_processLogs) (191001) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-22 09:17:22,468::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-22 09:17:22,469::__init__::225::IOProcess::(_processLogs) (191002) Finding callback 'access'... >Thread-14::DEBUG::2014-08-22 09:17:22,469::__init__::225::IOProcess::(_processLogs) (191002) Got request for method 'access' >Thread-14::DEBUG::2014-08-22 09:17:22,469::__init__::225::IOProcess::(_processLogs) (191002) Building response >Thread-14::DEBUG::2014-08-22 09:17:22,469::__init__::225::IOProcess::(_processLogs) (191002) Queuing response >Thread-14::DEBUG::2014-08-22 09:17:22,469::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-14::DEBUG::2014-08-22 09:17:22,469::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-13::DEBUG::2014-08-22 09:17:22,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`42d4d05c-49bc-4809-9904-6fa935ed5554`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:17:22,586::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-08-22 09:17:22,586::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000697801', 'lastCheck': '4.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000252658', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:17:22,586::task::1191::Storage.TaskManager.Task::(prepare) Task=`42d4d05c-49bc-4809-9904-6fa935ed5554`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000697801', 'lastCheck': '4.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000252658', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-08-22 09:17:22,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`42d4d05c-49bc-4809-9904-6fa935ed5554`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:17:22,587::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:17:22,587::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:17:22,587::task::993::Storage.TaskManager.Task::(_decref) Task=`42d4d05c-49bc-4809-9904-6fa935ed5554`::ref 0 aborting False >Thread-13::DEBUG::2014-08-22 09:17:22,610::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Domain Metadata is not set >MainThread::INFO::2014-08-22 09:17:24,720::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-08-22 09:17:24,720::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-91::DEBUG::2014-08-22 09:17:24,720::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 7c4cb940-6043-48fa-88f8-ab4762019c07 >MainThread::INFO::2014-08-22 09:17:24,720::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-16::DEBUG::2014-08-22 09:17:24,720::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-16::INFO::2014-08-22 09:17:24,720::clusterlock::206::Storage.SANLock::(releaseHostId) Releasing host id for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 (id: 2) >Thread-16::DEBUG::2014-08-22 09:17:24,721::domainMonitor::184::Storage.DomainMonitorThread::(_monitorLoop) Unable to release the host id 2 for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 181, in _monitorLoop > self.domain.releaseHostId(self.hostId, unused=True) > File "/usr/share/vdsm/storage/sd.py", line 480, in releaseHostId > self._clusterLock.releaseHostId(hostId, async, unused) > File "/usr/share/vdsm/storage/clusterlock.py", line 213, in releaseHostId > raise se.ReleaseHostIdFailure(self._sdUUID, e) >ReleaseHostIdFailure: Cannot release host id: (u'49f79085-53a0-4d45-8e3e-610bdf644da6', SanlockException(16, 'Sanlock lockspace remove failure', 'Device or resource busy')) >MainThread::DEBUG::2014-08-22 09:17:24,740::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-08-22 09:17:24,740::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-08-22 09:17:24,740::task::1191::Storage.TaskManager.Task::(prepare) Task=`8081b024-0e57-451c-ad5c-dd952c3f994e`::finished: None >MainThread::DEBUG::2014-08-22 09:17:24,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`8081b024-0e57-451c-ad5c-dd952c3f994e`::moving from state preparing -> state finished >MainThread::DEBUG::2014-08-22 09:17:24,740::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-08-22 09:17:24,740::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-08-22 09:17:24,740::task::993::Storage.TaskManager.Task::(_decref) Task=`8081b024-0e57-451c-ad5c-dd952c3f994e`::ref 0 aborting False >MainThread::INFO::2014-08-22 09:17:24,740::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 19 other threads... >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <_MainThread(MainThread, started 140459595581184)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140459503122176)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <_DummyThread(Dummy-170, started daemon 140456740505344)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140458785240832)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140459255002880)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140459471652608)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140459202553600)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140459213043456)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140459223533312)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(b220296d-8fed-4c72-8048-e57f22351157, started daemon 140459482142464)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140459244513024)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140459234023168)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <Thread(Thread-172, started daemon 140456302212864)> >MainThread::INFO::2014-08-22 09:17:24,741::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140459492632320)> >MainThread::DEBUG::2014-08-22 09:17:24,742::sampling::406::vm.Vm::(stop) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Stop statistics collection >MainThread::INFO::2014-08-22 09:17:24,742::vdsm::141::vds::(run) <VmStatsThread(Thread-183, started daemon 140457220757248)> >Thread-183::DEBUG::2014-08-22 09:17:24,742::sampling::437::vm.Vm::(run) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Stats thread finished >MainThread::INFO::2014-08-22 09:17:24,742::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140457711499008)> >MainThread::INFO::2014-08-22 09:17:24,742::vdsm::141::vds::(run) <Thread(Thread-14, started daemon 140458764261120)> >MainThread::INFO::2014-08-22 09:17:24,742::vdsm::141::vds::(run) <Thread(Thread-171, started daemon 140456730015488)> >MainThread::INFO::2014-08-22 09:17:24,742::vdsm::141::vds::(run) <Thread(Thread-13, started daemon 140458128811776)> >MainThread::INFO::2014-08-22 09:17:24,742::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-08-22 09:17:24,742::vdsm::141::vds::(run) <MomThread(MOM, started 140458436519680)> >Thread-13::DEBUG::2014-08-22 09:17:29,203::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-13::DEBUG::2014-08-22 09:17:29,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c7c9a9b-e4a7-4e67-9c2f-2608ec3c5a8c`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:17:29,203::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-13::INFO::2014-08-22 09:17:29,203::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:17:29,204::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c7c9a9b-e4a7-4e67-9c2f-2608ec3c5a8c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 6}} >Thread-13::DEBUG::2014-08-22 09:17:29,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c7c9a9b-e4a7-4e67-9c2f-2608ec3c5a8c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-08-22 09:17:29,204::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-08-22 09:17:29,204::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-08-22 09:17:29,204::task::993::Storage.TaskManager.Task::(_decref) Task=`0c7c9a9b-e4a7-4e67-9c2f-2608ec3c5a8c`::ref 0 aborting False >Thread-171::DEBUG::2014-08-22 09:17:29,273::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-171::DEBUG::2014-08-22 09:17:29,273::task::595::Storage.TaskManager.Task::(_updateState) Task=`f028c46d-9275-4e86-a16f-8eb8654463b0`::moving from state init -> state preparing >Thread-171::INFO::2014-08-22 09:17:29,273::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-171::DEBUG::2014-08-22 09:17:29,273::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5b5ddde3-0cd9-4379-82f9-aca3e991dd6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-171::DEBUG::2014-08-22 09:17:29,274::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-171::DEBUG::2014-08-22 09:17:29,274::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-171::DEBUG::2014-08-22 09:17:29,274::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5b5ddde3-0cd9-4379-82f9-aca3e991dd6c`::Granted request >Thread-171::DEBUG::2014-08-22 09:17:29,274::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f028c46d-9275-4e86-a16f-8eb8654463b0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-171::DEBUG::2014-08-22 09:17:29,274::task::993::Storage.TaskManager.Task::(_decref) Task=`f028c46d-9275-4e86-a16f-8eb8654463b0`::ref 1 aborting False >Thread-171::INFO::2014-08-22 09:17:29,276::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': '', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'isoprefix': '', 'alerts': []}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'isoprefix': '', 'alerts': []}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:17:29,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`f028c46d-9275-4e86-a16f-8eb8654463b0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': '', 'pool_status': 'connected', 'lver': 6, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'isoprefix': '', 'alerts': []}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'isoprefix': '', 'alerts': []}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-171::DEBUG::2014-08-22 09:17:29,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`f028c46d-9275-4e86-a16f-8eb8654463b0`::moving from state preparing -> state finished >Thread-171::DEBUG::2014-08-22 09:17:29,276::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-171::DEBUG::2014-08-22 09:17:29,276::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2014-08-22 09:17:29,276::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-171::DEBUG::2014-08-22 09:17:29,276::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-171::DEBUG::2014-08-22 09:17:29,276::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-171::DEBUG::2014-08-22 09:17:29,276::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-171::DEBUG::2014-08-22 09:17:29,276::task::993::Storage.TaskManager.Task::(_decref) Task=`f028c46d-9275-4e86-a16f-8eb8654463b0`::ref 0 aborting False >MainThread::INFO::2014-08-22 09:17:32,630::vdsm::131::vds::(run) (PID: 10968) I am the actual vdsm 4.16.1-6.gita4a4614.el6 dell-r210ii-12.rhev.lab.eng.brq.redhat.com (2.6.32-431.el6.x86_64) >MainThread::DEBUG::2014-08-22 09:17:32,631::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-08-22 09:17:32,631::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-08-22 09:17:32,632::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-08-22 09:17:32,633::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-08-22 09:17:32,677::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-08-22 09:17:32,687::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-08-22 09:17:32,687::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-08-22 09:17:32,789::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-08-22 09:17:32,789::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-08-22 09:17:32,791::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-08-22 09:17:32,792::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01', '/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso'] >MainThread::DEBUG::2014-08-22 09:17:32,792::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-08-22 09:17:32,793::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-08-22 09:17:32,795::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-08-22 09:17:32,796::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-08-22 09:17:32,796::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-08-22 09:17:32,836::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-08-22 09:17:32,836::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-08-22 09:17:32,836::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >MainThread::DEBUG::2014-08-22 09:17:32,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1457c3f-3f8c-4b3c-b0e3-04119ac26f10`::moving from state init -> state preparing >storageRefresh::DEBUG::2014-08-22 09:17:32,839::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::INFO::2014-08-22 09:17:32,839::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x2a90d50>>) >MainThread::INFO::2014-08-22 09:17:32,842::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-08-22 09:17:32,843::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1457c3f-3f8c-4b3c-b0e3-04119ac26f10`::finished: None >MainThread::DEBUG::2014-08-22 09:17:32,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1457c3f-3f8c-4b3c-b0e3-04119ac26f10`::moving from state preparing -> state finished >MainThread::DEBUG::2014-08-22 09:17:32,843::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-08-22 09:17:32,843::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-08-22 09:17:32,843::task::993::Storage.TaskManager.Task::(_decref) Task=`a1457c3f-3f8c-4b3c-b0e3-04119ac26f10`::ref 0 aborting False >MainThread::INFO::2014-08-22 09:17:32,843::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-08-22 09:17:32,845::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >clientIFinit::DEBUG::2014-08-22 09:17:32,848::libvirtconnection::150::root::(get) trying to connect libvirt >VM Channels Listener::INFO::2014-08-22 09:17:32,848::vmchannels::175::vds::(run) Starting VM channels listener thread. >storageRefresh::DEBUG::2014-08-22 09:17:32,849::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-08-22 09:17:32,852::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::DEBUG::2014-08-22 09:17:32,863::protocoldetector::129::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x2bc23b0> >MainThread::DEBUG::2014-08-22 09:17:32,926::protocoldetector::129::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x2d72488> >JsonRpcServer::DEBUG::2014-08-22 09:17:32,929::__init__::498::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-08-22 09:17:32,931::protocoldetector::68::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-08-22 09:17:32,931::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=11 >clientIFinit::DEBUG::2014-08-22 09:17:32,944::clientIF::545::vds::(_recoverVm) Trying to recover 61901e42-5ccc-43da-aaa7-60ed6bfbd5f5 >clientIFinit::INFO::2014-08-22 09:17:32,948::clientIF::391::vds::(createVm) vmContainerLock acquired by vm 61901e42-5ccc-43da-aaa7-60ed6bfbd5f5 >storageRefresh::DEBUG::2014-08-22 09:17:32,958::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >clientIFinit::DEBUG::2014-08-22 09:17:32,959::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-08-22 09:17:32,961::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,963::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,964::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,965::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,966::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,966::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,967::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-08-22 09:17:32,968::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:32,973::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >clientIFinit::DEBUG::2014-08-22 09:17:32,997::clientIF::404::vds::(createVm) Total desktops after creation of 61901e42-5ccc-43da-aaa7-60ed6bfbd5f5 is 1 >Thread-13::DEBUG::2014-08-22 09:17:32,997::vm::2228::vm.Vm::(_startUnderlyingVm) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Start >Thread-13::DEBUG::2014-08-22 09:17:32,998::vm::2232::vm.Vm::(_startUnderlyingVm) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::_ongoingCreations acquired >Thread-13::INFO::2014-08-22 09:17:32,999::vm::3224::vm.Vm::(_run) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::VM wrapper has started >Thread-13::WARNING::2014-08-22 09:17:33,000::vm::2020::vm.Vm::(buildConfDevices) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found >Thread-13::WARNING::2014-08-22 09:17:33,001::vm::2020::vm.Vm::(buildConfDevices) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found >Thread-13::WARNING::2014-08-22 09:17:33,001::vm::2020::vm.Vm::(buildConfDevices) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unknown type found, device: '{'device': u'spicevmc', 'alias': u'channel2', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'3'}}' found >Thread-13::DEBUG::2014-08-22 09:17:33,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f0468a9-5ca7-4e0e-a76f-dd239afec458`::moving from state init -> state preparing >Thread-13::INFO::2014-08-22 09:17:33,003::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-13::ERROR::2014-08-22 09:17:33,004::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-13::ERROR::2014-08-22 09:17:33,004::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-13::DEBUG::2014-08-22 09:17:33,005::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-13::DEBUG::2014-08-22 09:17:33,006::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 49f79085-53a0-4d45-8e3e-610bdf644da6 (cwd None) >Thread-13::DEBUG::2014-08-22 09:17:33,095::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "49f79085-53a0-4d45-8e3e-610bdf644da6" not found\n Skipping volume group 49f79085-53a0-4d45-8e3e-610bdf644da6\n'; <rc> = 5 >Thread-13::WARNING::2014-08-22 09:17:33,097::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "49f79085-53a0-4d45-8e3e-610bdf644da6" not found', ' Skipping volume group 49f79085-53a0-4d45-8e3e-610bdf644da6'] >storageRefresh::DEBUG::2014-08-22 09:17:33,098::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-08-22 09:17:33,100::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:33,102::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:33,103::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-15::DEBUG::2014-08-22 09:17:33,109::__init__::157::IOProcessClient::(_run) Starting IOProcess... >Thread-17::INFO::2014-08-22 09:17:33,115::__init__::227::IOProcess::(_processLogs) Starting ioprocess >Thread-17::DEBUG::2014-08-22 09:17:33,116::__init__::225::IOProcess::(_processLogs) Closing unrelated FDs... >Thread-17::DEBUG::2014-08-22 09:17:33,117::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 0 (pipe:[126613610]) >Thread-17::DEBUG::2014-08-22 09:17:33,117::__init__::225::IOProcess::(_processLogs) Not closing FD 1 because it's in whitelist >Thread-17::DEBUG::2014-08-22 09:17:33,118::__init__::225::IOProcess::(_processLogs) Not closing FD 2 because it's in whitelist >Thread-17::DEBUG::2014-08-22 09:17:33,118::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 3 (socket:[126613137]) >Thread-17::DEBUG::2014-08-22 09:17:33,119::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 4 (/var/log/vdsm/vdsm.log) >Thread-17::DEBUG::2014-08-22 09:17:33,119::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 5 (/var/log/vdsm/connectivity.log) >Thread-17::DEBUG::2014-08-22 09:17:33,120::__init__::225::IOProcess::(_processLogs) Not closing FD 6 because it's the directory fd >Thread-17::DEBUG::2014-08-22 09:17:33,120::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 10 (/sys/kernel/mm/ksm/pages_to_scan) >Thread-17::DEBUG::2014-08-22 09:17:33,120::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 11 (socket:[126613150]/pages_to_scan) >Thread-17::DEBUG::2014-08-22 09:17:33,121::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 13 (/sys/kernel/mm/ksm/pages_volatile) >Thread-17::DEBUG::2014-08-22 09:17:33,121::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 14 ([eventpoll]/mm/ksm/pages_volatile) >Thread-17::DEBUG::2014-08-22 09:17:33,122::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 15 (/sys/kernel/mm/ksm/sleep_millisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,122::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 16 (socket:[126613207]/sleep_millisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,123::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 17 (/var/log/vdsm/mom.logeep_millisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,123::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 18 (/proc/cpuinfo/mom.logeep_millisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,124::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 19 (/proc/meminfo/mom.logeep_millisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,124::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 20 (/proc/vmstato/mom.logeep_millisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,125::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 21 (/sys/kernel/mm/ksm/full_scansisecs) >Thread-17::DEBUG::2014-08-22 09:17:33,125::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 22 (/sys/kernel/mm/ksm/pages_sharingcs) >Thread-17::DEBUG::2014-08-22 09:17:33,126::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 23 (/sys/kernel/mm/ksm/pages_unshareds) >Thread-17::DEBUG::2014-08-22 09:17:33,126::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 24 (/sys/kernel/mm/ksm/runes_unshareds) >Thread-17::DEBUG::2014-08-22 09:17:33,126::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 25 (/sys/kernel/mm/ksm/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,127::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 28 (socket:[126613326]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,127::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 29 (socket:[126613257]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,128::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 37 (pipe:[126613607]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,128::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 38 (pipe:[126613345]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,129::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 39 (pipe:[126613345]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,129::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 40 (pipe:[126613607]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,130::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 41 (pipe:[126613555]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,131::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 42 (pipe:[126613608]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,131::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 43 (pipe:[126613556]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,132::__init__::225::IOProcess::(_processLogs) Not closing FD 44 because it's in whitelist >Thread-17::DEBUG::2014-08-22 09:17:33,132::__init__::225::IOProcess::(_processLogs) Not closing FD 45 because it's in whitelist >Thread-17::DEBUG::2014-08-22 09:17:33,132::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 46 (pipe:[126613609]7]/pages_sharededs) >Thread-17::DEBUG::2014-08-22 09:17:33,133::__init__::225::IOProcess::(_processLogs) Opening communication channels... >Thread-17::DEBUG::2014-08-22 09:17:33,133::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,134::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 42 (pipe:[126613608]7]/paDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,135::__init__::225::IOProcess::(_processLogs) Message size is 138 >Thread-17::DEBUG::2014-08-22 09:17:33,135::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,136::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 42 (pipe:[126613608]7]/paDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,137::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,138::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:33,138::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:33,139::__init__::225::IOProcess::(_processLogs) (1) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:33,139::__init__::225::IOProcess::(_processLogs) (1) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:33,140::__init__::225::IOProcess::(_processLogs) (1) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,140::__init__::225::IOProcess::(_processLogs) (1) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,141::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,141::__init__::225::IOProcess::(_processLogs) Sending response sized 188 >Thread-17::DEBUG::2014-08-22 09:17:33,142::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 42 (pipe:[126613608]7]/paDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,142::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:17:33,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,143::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,145::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 42 (pipe:[126613608]7]/paDEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,146::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:33,147::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:33,148::__init__::225::IOProcess::(_processLogs) (2) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:33,149::__init__::225::IOProcess::(_processLogs) (2) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:33,150::__init__::225::IOProcess::(_processLogs) (2) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,152::__init__::225::IOProcess::(_processLogs) (2) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,153::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,154::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-13::DEBUG::2014-08-22 09:17:33,159::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:17:33,161::__init__::225::IOProcess::(_processLogs) Closing unrelated fd no: 42 (pipe:[126613608]7]/paDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,161::__init__::225::IOProcess::(_processLogs) Message size is 169 >Thread-17::DEBUG::2014-08-22 09:17:33,162::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,163::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,163::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,164::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:33,165::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:33,165::__init__::225::IOProcess::(_processLogs) (3) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:17:33,166::__init__::225::IOProcess::(_processLogs) (3) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:17:33,179::__init__::225::IOProcess::(_processLogs) DEBUG|(3) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,180::__init__::225::IOProcess::(_processLogs) (3) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,181::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,182::__init__::225::IOProcess::(_processLogs) Sending response sized 71 >Thread-13::DEBUG::2014-08-22 09:17:33,184::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:17:33,185::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,186::__init__::225::IOProcess::(_processLogs) Message size is 193 >Thread-17::DEBUG::2014-08-22 09:17:33,187::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,187::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,188::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,190::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-13::DEBUG::2014-08-22 09:17:33,190::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:33,191::__init__::225::IOProcess::(_processLogs) Extracting request information... >storageRefresh::DEBUG::2014-08-22 09:17:33,192::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:33,192::__init__::225::IOProcess::(_processLogs) (4) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:33,194::__init__::225::IOProcess::(_processLogs) (4) Got request for method 'readfile' >storageRefresh::DEBUG::2014-08-22 09:17:33,194::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-17::DEBUG::2014-08-22 09:17:33,195::__init__::225::IOProcess::(_processLogs) (4) Building response >storageRefresh::DEBUG::2014-08-22 09:17:33,195::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:33,196::__init__::225::IOProcess::(_processLogs) (4) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,200::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,201::__init__::225::IOProcess::(_processLogs) Sending response sized 829 >Thread-17::DEBUG::2014-08-22 09:17:33,203::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,204::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:17:33,205::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,205::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,206::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,207::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:33,207::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:33,208::__init__::225::IOProcess::(_processLogs) (5) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:33,208::__init__::225::IOProcess::(_processLogs) (5) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:33,209::__init__::225::IOProcess::(_processLogs) (5) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,209::__init__::225::IOProcess::(_processLogs) (5) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,210::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,210::__init__::225::IOProcess::(_processLogs) Sending response sized 71 >Thread-17::DEBUG::2014-08-22 09:17:33,211::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,211::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:17:33,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-13::DEBUG::2014-08-22 09:17:33,213::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:17:33,213::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,214::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-13::DEBUG::2014-08-22 09:17:33,214::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS' >Thread-17::DEBUG::2014-08-22 09:17:33,215::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-13::DEBUG::2014-08-22 09:17:33,216::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS' >Thread-17::DEBUG::2014-08-22 09:17:33,216::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:33,217::__init__::225::IOProcess::(_processLogs) (6) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:33,218::__init__::225::IOProcess::(_processLogs) (6) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:33,218::__init__::225::IOProcess::(_processLogs) (6) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,219::__init__::225::IOProcess::(_processLogs) (6) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,219::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,219::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:17:33,220::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,221::__init__::225::IOProcess::(_processLogs) Message size is 238 >Thread-17::DEBUG::2014-08-22 09:17:33,221::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,222::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:33,224::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:33,224::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:33,225::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:33,225::__init__::225::IOProcess::(_processLogs) (7) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:17:33,226::__init__::225::IOProcess::(_processLogs) (7) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:17:33,226::__init__::225::IOProcess::(_processLogs) (7) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,227::__init__::225::IOProcess::(_processLogs) (7) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,227::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,228::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:17:33,228::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:33,229::__init__::225::IOProcess::(_processLogs) Message size is 238 >Thread-17::DEBUG::2014-08-22 09:17:33,229::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:33,231::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Queuing request... >Thread-13::INFO::2014-08-22 09:17:33,231::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-17::DEBUG::2014-08-22 09:17:33,232::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-13::DEBUG::2014-08-22 09:17:33,232::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f0468a9-5ca7-4e0e-a76f-dd239afec458`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-17::DEBUG::2014-08-22 09:17:33,233::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-13::DEBUG::2014-08-22 09:17:33,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f0468a9-5ca7-4e0e-a76f-dd239afec458`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-08-22 09:17:33,234::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-13::DEBUG::2014-08-22 09:17:33,234::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-08-22 09:17:33,235::__init__::225::IOProcess::(_processLogs) (8) Finding callback 'stat'... >Thread-13::DEBUG::2014-08-22 09:17:33,235::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-08-22 09:17:33,236::__init__::225::IOProcess::(_processLogs) (8) Got request for method 'stat' >Thread-13::DEBUG::2014-08-22 09:17:33,236::task::993::Storage.TaskManager.Task::(_decref) Task=`1f0468a9-5ca7-4e0e-a76f-dd239afec458`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:17:33,236::__init__::225::IOProcess::(_processLogs) (8) Building response >Thread-17::DEBUG::2014-08-22 09:17:33,238::__init__::225::IOProcess::(_processLogs) (8) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:33,239::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:33,239::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-13::DEBUG::2014-08-22 09:17:33,243::vm::1035::vm.Vm::(__init__) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Ignoring param (target, 3145728) in BalloonDevice >storageRefresh::DEBUG::2014-08-22 09:17:33,283::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-08-22 09:17:33,285::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:33,287::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-13::DEBUG::2014-08-22 09:17:33,338::vm::5190::vm.Vm::(_getUnderlyingDriveInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-13::DEBUG::2014-08-22 09:17:33,339::vm::5211::vm.Vm::(_getUnderlyingDriveInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-13::DEBUG::2014-08-22 09:17:33,340::vm::5230::vm.Vm::(_getUnderlyingDriveInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-13::DEBUG::2014-08-22 09:17:33,342::vm::5190::vm.Vm::(_getUnderlyingDriveInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Looking for drive with attributes {'name': u'vda', 'bootOrder': '', 'boot': [], 'readonly': False, 'address': {u'slot': u'0x07', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'type': u'disk'} >Thread-13::DEBUG::2014-08-22 09:17:33,342::vm::5211::vm.Vm::(_getUnderlyingDriveInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Matched {'name': (u'vda', u'vda'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (False, False), 'address': ({u'slot': u'0x07', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x07', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', u'/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade'), 'type': (u'disk', u'disk')} >Thread-13::DEBUG::2014-08-22 09:17:33,343::vm::5230::vm.Vm::(_getUnderlyingDriveInfo) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Matched {'name': (u'vda', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (False, None), 'address': ({u'slot': u'0x07', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', None), 'type': (u'disk', None)} >storageRefresh::DEBUG::2014-08-22 09:17:33,386::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-08-22 09:17:33,388::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-08-22 09:17:33,390::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Thread-13::DEBUG::2014-08-22 09:17:33,702::vm::1357::vm.Vm::(blockDev) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unable to determine if the path '/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade' is a block device >Traceback (most recent call last): > File "/usr/share/vdsm/virt/vm.py", line 1354, in blockDev > self._blockDev = utils.isBlockDevice(self.path) > File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 104, in isBlockDevice > return stat.S_ISBLK(os.stat(path).st_mode) >OSError: [Errno 2] No such file or directory: '/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade' >Thread-13::INFO::2014-08-22 09:17:33,703::vm::2251::vm.Vm::(_startUnderlyingVm) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Skipping errors on recovery >Traceback (most recent call last): > File "/usr/share/vdsm/virt/vm.py", line 2235, in _startUnderlyingVm > self._run() > File "/usr/share/vdsm/virt/vm.py", line 3298, in _run > self._domDependentInit() > File "/usr/share/vdsm/virt/vm.py", line 3189, in _domDependentInit > self._syncVolumeChain(drive) > File "/usr/share/vdsm/virt/vm.py", line 5660, in _syncVolumeChain > volumes = self._driveGetActualVolumeChain(drive) > File "/usr/share/vdsm/virt/vm.py", line 5639, in _driveGetActualVolumeChain > sourceAttr = ('file', 'dev')[drive.blockDev] >TypeError: tuple indices must be integers, not NoneType >Thread-13::DEBUG::2014-08-22 09:17:33,704::vm::2254::vm.Vm::(_startUnderlyingVm) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::_ongoingCreations released >clientIFinit::DEBUG::2014-08-22 09:17:34,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f49eb85-6e1d-43b1-94ed-9a79d00b54b5`::moving from state init -> state preparing >clientIFinit::INFO::2014-08-22 09:17:34,000::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-08-22 09:17:34,000::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-08-22 09:17:34,000::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f49eb85-6e1d-43b1-94ed-9a79d00b54b5`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-08-22 09:17:34,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f49eb85-6e1d-43b1-94ed-9a79d00b54b5`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-08-22 09:17:34,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-08-22 09:17:34,001::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-08-22 09:17:34,001::task::993::Storage.TaskManager.Task::(_decref) Task=`4f49eb85-6e1d-43b1-94ed-9a79d00b54b5`::ref 0 aborting False >Detector thread::DEBUG::2014-08-22 09:17:35,010::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.60:53201 >Detector thread::DEBUG::2014-08-22 09:17:35,011::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.60:53201 >Detector thread::DEBUG::2014-08-22 09:17:35,011::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.34.63.60:53201 >Detector thread::DEBUG::2014-08-22 09:17:35,011::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.34.63.60', 53201) >Thread-18::DEBUG::2014-08-22 09:17:35,012::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call getCapabilities with () {} >Thread-18::DEBUG::2014-08-22 09:17:35,050::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,057::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,072::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,081::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,082::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,092::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,094::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,102::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,103::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,112::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,115::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,123::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,124::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,132::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,133::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,141::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,142::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Ylqui1r0uvUpfRk/bonding/mode (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,149::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,151::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,186::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,206::caps::614::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.11 required >= 0.10.2-31 >Thread-18::DEBUG::2014-08-22 09:17:35,207::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,211::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,211::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,215::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,221::BindingXMLRPC::1134::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:24a1a947af1b'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.el6.x86_64', 'buildtime': 1384118394.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '6.gita4a4614.el6', 'buildtime': 1407224890L, 'version': '4.16.1'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.11', 'buildtime': 1406015335L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'mom': {'release': '2.el6', 'buildtime': 1403794344L, 'version': '0.4.1'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz', 'hooks': {}, 'cpuSockets': '1', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 1, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'iface': 'ovirtmgmt', 'addr': '10.34.62.204', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'no'}, 'bridged': True, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:17f6/64', 'fe80::d6ae:52ff:fec7:17f6/64'], 'gateway': '10.34.63.254', 'bootproto4': 'dhcp', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.204/22'], 'mtu': '1500', 'ipv6gateway': 'fe80:52:0:223c::3fe', 'ports': ['em1', 'vnet0']}}, 'bridges': {'ovirtmgmt': {'addr': '10.34.62.204', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'no'}, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:17f6/64', 'fe80::d6ae:52ff:fec7:17f6/64'], 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.204/22'], 'ipv6gateway': 'fe80:52:0:223c::3fe', 'gateway': '10.34.63.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '108', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.d4ae52c717f6', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.d4ae52c717f6', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '8', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['em1', 'vnet0']}}, 'uuid': '4C4C4544-0034-5310-8052-B1C04F4A354A', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'addr': '', 'cfg': {'BRIDGE': 'ovirtmgmt', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': 'd4:ae:52:c7:17:f6', 'DEVICE': 'em1', 'ONBOOT': 'no'}, 'ipv6addrs': ['fe80::d6ae:52ff:fec7:17f6/64'], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:17:f6', 'speed': 1000}, 'em2': {'addr': '', 'cfg': {'UUID': '054d296d-726c-4384-b993-4dc01d90c96e', 'NM_CONTROLLED': 'yes', 'HWADDR': 'D4:AE:52:C7:17:F7', 'BOOTPROTO': 'dhcp', 'DEVICE': 'em2', 'TYPE': 'Ethernet', 'ONBOOT': 'no'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:17:f7', 'speed': 0}}, 'software_revision': '6', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:24a1a947af1b', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond2': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond3': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}}, 'software_version': '4.16', 'memSize': '7842', 'cpuSpeed': '3301.000', 'numaNodes': {u'0': {'totalMemory': '7842', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}, 'lastClient': '10.34.63.60'}} >Thread-18::DEBUG::2014-08-22 09:17:35,243::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call getHardwareInfo with () {} >Thread-18::DEBUG::2014-08-22 09:17:35,245::BindingXMLRPC::1134::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'PowerEdge R210 II', 'systemUUID': '4C4C4544-0034-5310-8052-B1C04F4A354A', 'systemSerialNumber': '14SRJ5J', 'systemManufacturer': 'Dell Inc.'}} >Thread-18::DEBUG::2014-08-22 09:17:35,492::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call vmGetStats with ('61901e42-5ccc-43da-aaa7-60ed6bfbd5f5',) {} flowID [73f1b5c0] >Thread-18::DEBUG::2014-08-22 09:17:35,492::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Paused', 'displayInfo': [{'tlsPort': u'5901', 'ipAddress': '10.34.62.204', 'type': 'spice', 'port': u'5900'}], 'hash': '-5479945782673424400', 'acpiEnable': 'true', 'pid': '13420', 'displayIp': '10.34.62.204', 'displayPort': u'5900', 'displaySecurePort': u'5901', 'timeOffset': '0', 'kvmEnable': 'true', 'network': {}, 'vmId': '61901e42-5ccc-43da-aaa7-60ed6bfbd5f5', 'vmType': 'kvm', 'disks': {}, 'monitorResponse': '0', 'elapsedTime': '234500', 'displayType': 'qxl', 'clientIp': ''}]} >Thread-18::DEBUG::2014-08-22 09:17:35,695::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [76cc9e11] >Thread-18::DEBUG::2014-08-22 09:17:35,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`39ab3a68-f02b-40e5-ac3a-4cb90c3eda37`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:35,695::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', conList=[{'connection': '10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '21f1184f-938f-4ce5-beac-3606c28e9453', 'port': ''}, {'connection': '10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '78e71ca7-6a3f-4718-a4e2-da0b902af70d', 'port': ''}], options=None) >Thread-18::DEBUG::2014-08-22 09:17:35,697::hsm::2366::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01 >Thread-17::DEBUG::2014-08-22 09:17:35,697::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,697::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) (9) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) (9) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,698::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|(9) Building response >Thread-18::DEBUG::2014-08-22 09:17:35,699::hsm::2390::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'49f79085-53a0-4d45-8e3e-610bdf644da6',) >Thread-17::DEBUG::2014-08-22 09:17:35,699::__init__::225::IOProcess::(_processLogs) (9) Queuing response >Thread-18::DEBUG::2014-08-22 09:17:35,699::hsm::2446::Storage.HSM::(connectStorageServer) knownSDs: {49f79085-53a0-4d45-8e3e-610bdf644da6: storage.nfsSD.findDomain} >Thread-17::DEBUG::2014-08-22 09:17:35,700::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,700::__init__::225::IOProcess::(_processLogs) Sending response sized 183 >Thread-18::DEBUG::2014-08-22 09:17:35,702::hsm::2366::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso >Thread-17::DEBUG::2014-08-22 09:17:35,702::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,702::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:35,702::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,702::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,703::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,703::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,703::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,703::__init__::225::IOProcess::(_processLogs) (10) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,703::__init__::225::IOProcess::(_processLogs) (10) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,703::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|(10) Building response >Thread-18::DEBUG::2014-08-22 09:17:35,703::hsm::2390::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'7c4cb940-6043-48fa-88f8-ab4762019c07',) >Thread-17::DEBUG::2014-08-22 09:17:35,704::__init__::225::IOProcess::(_processLogs) (10) Queuing response >Thread-18::DEBUG::2014-08-22 09:17:35,704::hsm::2446::Storage.HSM::(connectStorageServer) knownSDs: {7c4cb940-6043-48fa-88f8-ab4762019c07: storage.nfsSD.findDomain, 49f79085-53a0-4d45-8e3e-610bdf644da6: storage.nfsSD.findDomain} >Thread-17::DEBUG::2014-08-22 09:17:35,704::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-18::INFO::2014-08-22 09:17:35,704::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '21f1184f-938f-4ce5-beac-3606c28e9453'}, {'status': 0, 'id': '78e71ca7-6a3f-4718-a4e2-da0b902af70d'}]} >Thread-17::DEBUG::2014-08-22 09:17:35,705::__init__::225::IOProcess::(_processLogs) Sending response sized 182 >Thread-18::DEBUG::2014-08-22 09:17:35,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`39ab3a68-f02b-40e5-ac3a-4cb90c3eda37`::finished: {'statuslist': [{'status': 0, 'id': '21f1184f-938f-4ce5-beac-3606c28e9453'}, {'status': 0, 'id': '78e71ca7-6a3f-4718-a4e2-da0b902af70d'}]} >Thread-18::DEBUG::2014-08-22 09:17:35,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`39ab3a68-f02b-40e5-ac3a-4cb90c3eda37`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:35,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:35,705::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:35,706::task::993::Storage.TaskManager.Task::(_decref) Task=`39ab3a68-f02b-40e5-ac3a-4cb90c3eda37`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:35,727::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:17:35,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`61789fe6-4d7e-4b16-b5af-62d44f5dfaa0`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:35,727::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', hostID=2, msdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', masterVersion=1, domainsMap=None, options=None) >Thread-18::DEBUG::2014-08-22 09:17:35,727::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`ed431416-8ee4-4ab8-86a0-b0caa421a3f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-18::DEBUG::2014-08-22 09:17:35,727::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-18::DEBUG::2014-08-22 09:17:35,728::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-18::DEBUG::2014-08-22 09:17:35,728::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`ed431416-8ee4-4ab8-86a0-b0caa421a3f2`::Granted request >Thread-18::DEBUG::2014-08-22 09:17:35,728::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b165a6a4-d9c7-4dfc-8e4e-70a787c4fbd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-18::DEBUG::2014-08-22 09:17:35,728::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'exclusive' >Thread-18::DEBUG::2014-08-22 09:17:35,728::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'exclusive' (1 active user) >Thread-18::DEBUG::2014-08-22 09:17:35,728::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b165a6a4-d9c7-4dfc-8e4e-70a787c4fbd4`::Granted request >Thread-18::INFO::2014-08-22 09:17:35,729::sp::133::Storage.StoragePool::(setBackend) updating pool da0e21b4-cba8-40bf-a5cd-3a50c2199246 backend from type NoneType instance 0x3c9d972210 to type StoragePoolDiskBackend instance 0x7f1f0c2a9c68 >Thread-18::INFO::2014-08-22 09:17:35,729::sp::631::Storage.StoragePool::(connect) Connect host #2 to the storage pool da0e21b4-cba8-40bf-a5cd-3a50c2199246 with master domain: 49f79085-53a0-4d45-8e3e-610bdf644da6 (ver = 1) >Thread-18::DEBUG::2014-08-22 09:17:35,729::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,729::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,729::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,729::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,729::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,729::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,729::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-18::DEBUG::2014-08-22 09:17:35,730::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-18::DEBUG::2014-08-22 09:17:35,730::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-18::DEBUG::2014-08-22 09:17:35,730::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-18::DEBUG::2014-08-22 09:17:35,730::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-18::DEBUG::2014-08-22 09:17:35,730::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,738::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-18::DEBUG::2014-08-22 09:17:35,739::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,797::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:35,798::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,798::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,798::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,798::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,798::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,798::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-08-22 09:17:35,798::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-17::DEBUG::2014-08-22 09:17:35,800::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,800::__init__::225::IOProcess::(_processLogs) Message size is 139 >Thread-17::DEBUG::2014-08-22 09:17:35,800::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,801::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,802::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,802::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,802::__init__::225::IOProcess::(_processLogs) (11) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,802::__init__::225::IOProcess::(_processLogs) (12) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,802::__init__::225::IOProcess::(_processLogs) DEBUG|(12) Got request for method 'glob'(11) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,802::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|(11) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) (11) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) Sending response sized 189 >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) (12) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) (12) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,803::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-18::DEBUG::2014-08-22 09:17:35,805::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) (13) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:17:35,806::__init__::225::IOProcess::(_processLogs) (13) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:17:35,825::__init__::225::IOProcess::(_processLogs) Queuing request...DEBUG|(13) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,825::__init__::225::IOProcess::(_processLogs) (13) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,825::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,825::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-18::DEBUG::2014-08-22 09:17:35,825::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:17:35,826::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:35,826::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,826::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,826::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,827::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,827::__init__::225::IOProcess::(_processLogs) (14) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:35,827::__init__::225::IOProcess::(_processLogs) (14) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:35,827::__init__::225::IOProcess::(_processLogs) (14) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,827::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-18::DEBUG::2014-08-22 09:17:35,827::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:35,828::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 09:17:35,828::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,829::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,829::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,829::__init__::225::IOProcess::(_processLogs) (15) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:35,829::__init__::225::IOProcess::(_processLogs) (15) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:35,829::__init__::225::IOProcess::(_processLogs) (15) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) (15) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) (16) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,830::__init__::225::IOProcess::(_processLogs) (16) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,831::__init__::225::IOProcess::(_processLogs) (16) Queuing response >Thread-18::DEBUG::2014-08-22 09:17:35,831::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:17:35,831::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-18::INFO::2014-08-22 09:17:35,831::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:17:35,832::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::INFO::2014-08-22 09:17:35,832::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-18::DEBUG::2014-08-22 09:17:35,833::sp::1387::Storage.StoragePool::(setMasterDomain) Master domain 49f79085-53a0-4d45-8e3e-610bdf644da6 verified, version 1 >Thread-18::DEBUG::2014-08-22 09:17:35,833::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-18::DEBUG::2014-08-22 09:17:35,833::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-17::DEBUG::2014-08-22 09:17:35,833::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:35,833::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,833::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,834::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,834::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,834::__init__::225::IOProcess::(_processLogs) (17) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:35,834::__init__::225::IOProcess::(_processLogs) (17) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:35,834::__init__::225::IOProcess::(_processLogs) (17) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,834::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-18::DEBUG::2014-08-22 09:17:35,834::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:35,835::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-18::INFO::2014-08-22 09:17:35,836::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-18::DEBUG::2014-08-22 09:17:35,836::sp::1410::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `da0e21b4-cba8-40bf-a5cd-3a50c2199246` started monitoring domain `7c4cb940-6043-48fa-88f8-ab4762019c07` >Thread-21::DEBUG::2014-08-22 09:17:35,836::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-18::INFO::2014-08-22 09:17:35,836::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-18::DEBUG::2014-08-22 09:17:35,838::sp::1410::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `da0e21b4-cba8-40bf-a5cd-3a50c2199246` started monitoring domain `49f79085-53a0-4d45-8e3e-610bdf644da6` >Thread-22::DEBUG::2014-08-22 09:17:35,838::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-18::DEBUG::2014-08-22 09:17:35,840::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-17::DEBUG::2014-08-22 09:17:35,841::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-18::DEBUG::2014-08-22 09:17:35,841::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246 >Thread-17::DEBUG::2014-08-22 09:17:35,841::__init__::225::IOProcess::(_processLogs) Message size is 139 >Thread-17::DEBUG::2014-08-22 09:17:35,841::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,842::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,843::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,843::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,844::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,844::__init__::225::IOProcess::(_processLogs) (18) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,844::__init__::225::IOProcess::(_processLogs) (18) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,845::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(18) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,845::__init__::225::IOProcess::(_processLogs) (18) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,846::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,846::__init__::225::IOProcess::(_processLogs) Sending response sized 189 >Thread-17::DEBUG::2014-08-22 09:17:35,848::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,848::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:17:35,849::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,850::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,850::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,851::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,853::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:17:35,854::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:17:35,854::__init__::225::IOProcess::(_processLogs) (19) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) (19) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) (19) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) (19) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,855::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,856::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,856::__init__::225::IOProcess::(_processLogs) (20) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:35,856::__init__::225::IOProcess::(_processLogs) (20) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:35,856::__init__::225::IOProcess::(_processLogs) (20) Queuing response >Thread-22::DEBUG::2014-08-22 09:17:35,857::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:35,857::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,861::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-22::DEBUG::2014-08-22 09:17:35,862::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00020116 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:35,862::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,863::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:35,863::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,863::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,863::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,863::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,863::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,864::__init__::225::IOProcess::(_processLogs) (21) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:35,864::__init__::225::IOProcess::(_processLogs) (21) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:35,864::__init__::225::IOProcess::(_processLogs) (21) Queuing response >Thread-18::DEBUG::2014-08-22 09:17:35,864::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:35,865::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,865::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-18::DEBUG::2014-08-22 09:17:35,866::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-17::DEBUG::2014-08-22 09:17:35,867::__init__::225::IOProcess::(_processLogs) Message size is 168 >Thread-17::DEBUG::2014-08-22 09:17:35,867::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,868::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-18::DEBUG::2014-08-22 09:17:35,868::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 7c4cb940-6043-48fa-88f8-ab4762019c07 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:35,868::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,872::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,872::__init__::225::IOProcess::(_processLogs) (22) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) (22) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) (23) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:35,873::__init__::225::IOProcess::(_processLogs) (23) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:35,874::__init__::225::IOProcess::(_processLogs) (23) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,875::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,875::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-17::DEBUG::2014-08-22 09:17:35,875::__init__::225::IOProcess::(_processLogs) Message size is 178 >Thread-17::DEBUG::2014-08-22 09:17:35,875::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) (24) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) (24) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) (24) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) (24) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,876::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:35,877::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:17:35,877::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,877::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,877::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,877::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,877::__init__::225::IOProcess::(_processLogs) (25) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) (25) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) (25) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) (25) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,878::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) (26) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) (26) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) (26) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) (26) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,879::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-22::DEBUG::2014-08-22 09:17:35,880::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain 49f79085-53a0-4d45-8e3e-610bdf644da6 changed its status to Valid >Thread-25::DEBUG::2014-08-22 09:17:35,880::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-25::DEBUG::2014-08-22 09:17:35,881::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-25::DEBUG::2014-08-22 09:17:35,881::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-26::INFO::2014-08-22 09:17:35,881::clientIF::137::vds::(contEIOVms) vmContainerLock acquired >Thread-17::DEBUG::2014-08-22 09:17:35,883::__init__::225::IOProcess::(_processLogs) (22) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,883::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:17:35,883::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:17:35,884::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:35,884::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) Message size is 192 >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) (27) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:35,885::__init__::225::IOProcess::(_processLogs) (27) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:35,916::__init__::225::IOProcess::(_processLogs) DEBUG|(27) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,916::__init__::225::IOProcess::(_processLogs) (27) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,916::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:17:35,916::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:17:35,917::__init__::225::IOProcess::(_processLogs) Sending response sized 602 >Thread-17::DEBUG::2014-08-22 09:17:35,918::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,918::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:17:35,918::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,918::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) (28) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) (28) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) (28) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) (28) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:35,919::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) (29) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(29) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,920::__init__::225::IOProcess::(_processLogs) (29) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,921::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:17:35,921::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:17:35,921::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-21::DEBUG::2014-08-22 09:17:35,922::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS' >Thread-21::DEBUG::2014-08-22 09:17:35,922::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS' >Thread-17::DEBUG::2014-08-22 09:17:35,922::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,922::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:35,923::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,923::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,924::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:17:35,924::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:35,924::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,929::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,929::__init__::225::IOProcess::(_processLogs) (30) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:35,929::__init__::225::IOProcess::(_processLogs) (30) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:35,929::__init__::225::IOProcess::(_processLogs) (30) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,929::__init__::225::IOProcess::(_processLogs) (30) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,929::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,930::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-21::DEBUG::2014-08-22 09:17:35,933::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00464446 s, 85.5 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:35,934::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,934::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:35,934::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,934::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,934::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,935::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Extracting request information... >Thread-21::DEBUG::2014-08-22 09:17:35,936::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain 7c4cb940-6043-48fa-88f8-ab4762019c07 changed its status to Valid >Thread-17::DEBUG::2014-08-22 09:17:35,936::__init__::225::IOProcess::(_processLogs) (31) Finding callback 'statvfs'... >Thread-27::DEBUG::2014-08-22 09:17:35,937::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-17::DEBUG::2014-08-22 09:17:35,937::__init__::225::IOProcess::(_processLogs) (31) Got request for method 'statvfs' >Thread-27::DEBUG::2014-08-22 09:17:35,937::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-17::DEBUG::2014-08-22 09:17:35,938::__init__::225::IOProcess::(_processLogs) (31) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,939::__init__::225::IOProcess::(_processLogs) (31) Queuing response >Thread-27::DEBUG::2014-08-22 09:17:35,938::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-17::DEBUG::2014-08-22 09:17:35,939::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,939::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-28::INFO::2014-08-22 09:17:35,940::clientIF::137::vds::(contEIOVms) vmContainerLock acquired >Thread-18::DEBUG::2014-08-22 09:17:35,961::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "7c4cb940-6043-48fa-88f8-ab4762019c07" not found\n Skipping volume group 7c4cb940-6043-48fa-88f8-ab4762019c07\n'; <rc> = 5 >Thread-18::WARNING::2014-08-22 09:17:35,962::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "7c4cb940-6043-48fa-88f8-ab4762019c07" not found', ' Skipping volume group 7c4cb940-6043-48fa-88f8-ab4762019c07'] >Thread-18::DEBUG::2014-08-22 09:17:35,963::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-17::DEBUG::2014-08-22 09:17:35,964::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,964::__init__::225::IOProcess::(_processLogs) Message size is 139 >Thread-17::DEBUG::2014-08-22 09:17:35,964::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) (32) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) (32) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) (32) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,965::__init__::225::IOProcess::(_processLogs) (32) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:35,966::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,966::__init__::225::IOProcess::(_processLogs) Sending response sized 189 >Thread-17::DEBUG::2014-08-22 09:17:35,966::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:35,966::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:17:35,967::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:35,967::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-18::INFO::2014-08-22 09:17:35,967::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 to /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:17:35,968::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:35,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-08-22 09:17:35,968::sp::1135::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 to /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:17:35,969::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-18::INFO::2014-08-22 09:17:35,969::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 to /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:17:35,970::__init__::225::IOProcess::(_processLogs) (33) Finding callback 'glob'... >Thread-18::DEBUG::2014-08-22 09:17:35,970::sp::1135::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 to /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:17:35,971::__init__::225::IOProcess::(_processLogs) (33) Got request for method 'glob' >Thread-18::INFO::2014-08-22 09:17:35,971::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 to /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd >Thread-17::DEBUG::2014-08-22 09:17:35,971::__init__::225::IOProcess::(_processLogs) (33) Building response >Thread-17::DEBUG::2014-08-22 09:17:35,972::__init__::225::IOProcess::(_processLogs) (33) Queuing response >Thread-18::DEBUG::2014-08-22 09:17:35,972::sp::1135::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 to /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd >Thread-17::DEBUG::2014-08-22 09:17:35,973::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:35,973::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-18::DEBUG::2014-08-22 09:17:35,974::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-18::DEBUG::2014-08-22 09:17:35,975::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=16 (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:35,985::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00513642 s, 797 kB/s\n'; <rc> = 0 >Thread-18::INFO::2014-08-22 09:17:35,985::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-18::DEBUG::2014-08-22 09:17:35,985::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-18::DEBUG::2014-08-22 09:17:36,053::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.0628956 s, 65.1 kB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:36,053::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool da0e21b4-cba8-40bf-a5cd-3a50c2199246 >Thread-18::DEBUG::2014-08-22 09:17:36,053::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool da0e21b4-cba8-40bf-a5cd-3a50c2199246 on master domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-18::DEBUG::2014-08-22 09:17:36,054::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-18::INFO::2014-08-22 09:17:36,055::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-18::DEBUG::2014-08-22 09:17:36,055::task::1191::Storage.TaskManager.Task::(prepare) Task=`61789fe6-4d7e-4b16-b5af-62d44f5dfaa0`::finished: True >Thread-18::DEBUG::2014-08-22 09:17:36,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`61789fe6-4d7e-4b16-b5af-62d44f5dfaa0`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:36,055::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:36,055::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:36,055::task::993::Storage.TaskManager.Task::(_decref) Task=`61789fe6-4d7e-4b16-b5af-62d44f5dfaa0`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:36,102::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e9813fb-38f2-44fa-95bc-0260dc68303d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:36,102::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:17:36,102::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00464446', 'lastCheck': '0.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020116', 'lastCheck': '0.2', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:17:36,102::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e9813fb-38f2-44fa-95bc-0260dc68303d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00464446', 'lastCheck': '0.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020116', 'lastCheck': '0.2', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:17:36,102::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e9813fb-38f2-44fa-95bc-0260dc68303d`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:36,102::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:36,102::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:36,102::task::993::Storage.TaskManager.Task::(_decref) Task=`9e9813fb-38f2-44fa-95bc-0260dc68303d`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:36,141::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call setMOMPolicyParameters with ({'balloonEnabled': False, 'ksmEnabled': False},) {} flowID [76cc9e11] >Thread-18::DEBUG::2014-08-22 09:17:36,190::BindingXMLRPC::1134::vds::(wrapper) return setMOMPolicyParameters with {'status': {'message': 'Done', 'code': 0}} >clientIFinit::DEBUG::2014-08-22 09:17:39,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c75f248-3212-47c5-8f65-422e0579179a`::moving from state init -> state preparing >clientIFinit::INFO::2014-08-22 09:17:39,006::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-08-22 09:17:39,006::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': ['da0e21b4-cba8-40bf-a5cd-3a50c2199246']} >clientIFinit::DEBUG::2014-08-22 09:17:39,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c75f248-3212-47c5-8f65-422e0579179a`::finished: {'poollist': ['da0e21b4-cba8-40bf-a5cd-3a50c2199246']} >clientIFinit::DEBUG::2014-08-22 09:17:39,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c75f248-3212-47c5-8f65-422e0579179a`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-08-22 09:17:39,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-08-22 09:17:39,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-08-22 09:17:39,007::task::993::Storage.TaskManager.Task::(_decref) Task=`6c75f248-3212-47c5-8f65-422e0579179a`::ref 0 aborting False >clientIFinit::WARNING::2014-08-22 09:17:39,007::vm::2020::vm.Vm::(buildConfDevices) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found >clientIFinit::WARNING::2014-08-22 09:17:39,007::vm::2020::vm.Vm::(buildConfDevices) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found >clientIFinit::WARNING::2014-08-22 09:17:39,007::vm::2020::vm.Vm::(buildConfDevices) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::Unknown type found, device: '{'device': u'spicevmc', 'alias': u'channel2', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'3'}}' found >clientIFinit::DEBUG::2014-08-22 09:17:39,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`6659cc11-d476-4b72-a009-f00b058c155b`::moving from state init -> state preparing >clientIFinit::INFO::2014-08-22 09:17:39,008::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', volUUID='f61ef654-8b79-40ad-ba12-f396a913cade', options=None) >Thread-17::DEBUG::2014-08-22 09:17:39,008::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,008::__init__::225::IOProcess::(_processLogs) Message size is 239 >Thread-17::DEBUG::2014-08-22 09:17:39,009::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,009::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,009::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,009::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,009::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) (34) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) (34) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) (34) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) (34) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) Sending response sized 330 >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) Message size is 239 >Thread-17::DEBUG::2014-08-22 09:17:39,010::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,011::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >clientIFinit::INFO::2014-08-22 09:17:39,011::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-17::DEBUG::2014-08-22 09:17:39,011::__init__::225::IOProcess::(_processLogs) Waiting for next request... >clientIFinit::DEBUG::2014-08-22 09:17:39,011::task::1191::Storage.TaskManager.Task::(prepare) Task=`6659cc11-d476-4b72-a009-f00b058c155b`::finished: {'truesize': '1689866240', 'apparentsize': '1689976832'} >Thread-17::DEBUG::2014-08-22 09:17:39,012::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >clientIFinit::DEBUG::2014-08-22 09:17:39,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`6659cc11-d476-4b72-a009-f00b058c155b`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-08-22 09:17:39,012::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-08-22 09:17:39,012::__init__::225::IOProcess::(_processLogs) Extracting request information... >clientIFinit::DEBUG::2014-08-22 09:17:39,013::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-08-22 09:17:39,013::__init__::225::IOProcess::(_processLogs) (35) Finding callback 'stat'... >clientIFinit::DEBUG::2014-08-22 09:17:39,013::task::993::Storage.TaskManager.Task::(_decref) Task=`6659cc11-d476-4b72-a009-f00b058c155b`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:17:39,013::__init__::225::IOProcess::(_processLogs) (35) Got request for method 'stat' >clientIFinit::INFO::2014-08-22 09:17:39,014::clientIF::332::vds::(prepareVolumePath) prepared volume path: >Thread-17::DEBUG::2014-08-22 09:17:39,014::__init__::225::IOProcess::(_processLogs) (35) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,015::__init__::225::IOProcess::(_processLogs) (35) Queuing response >clientIFinit::DEBUG::2014-08-22 09:17:39,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d809cf6-2f7a-4dd6-8f63-d0caf46495f4`::moving from state init -> state preparing >Thread-17::DEBUG::2014-08-22 09:17:39,015::__init__::225::IOProcess::(_processLogs) Generating json... >clientIFinit::INFO::2014-08-22 09:17:39,015::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='49f79085-53a0-4d45-8e3e-610bdf644da6', spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', imgUUID='b7b0a6c3-4adf-4a92-9132-00da6ae85621', leafUUID='f61ef654-8b79-40ad-ba12-f396a913cade') >Thread-17::DEBUG::2014-08-22 09:17:39,016::__init__::225::IOProcess::(_processLogs) Sending response sized 330 >clientIFinit::DEBUG::2014-08-22 09:17:39,016::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.49f79085-53a0-4d45-8e3e-610bdf644da6`ReqID=`8e67876e-ea8d-4b53-b95d-3e7aa2a43b12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3204' at 'prepareImage' >clientIFinit::DEBUG::2014-08-22 09:17:39,016::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' for lock type 'shared' >clientIFinit::DEBUG::2014-08-22 09:17:39,016::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' is free. Now locking as 'shared' (1 active user) >clientIFinit::DEBUG::2014-08-22 09:17:39,017::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.49f79085-53a0-4d45-8e3e-610bdf644da6`ReqID=`8e67876e-ea8d-4b53-b95d-3e7aa2a43b12`::Granted request >clientIFinit::DEBUG::2014-08-22 09:17:39,017::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9d809cf6-2f7a-4dd6-8f63-d0caf46495f4`::_resourcesAcquired: Storage.49f79085-53a0-4d45-8e3e-610bdf644da6 (shared) >clientIFinit::DEBUG::2014-08-22 09:17:39,017::task::993::Storage.TaskManager.Task::(_decref) Task=`9d809cf6-2f7a-4dd6-8f63-d0caf46495f4`::ref 1 aborting False >Thread-17::DEBUG::2014-08-22 09:17:39,017::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,017::__init__::225::IOProcess::(_processLogs) Message size is 177 >Thread-17::DEBUG::2014-08-22 09:17:39,017::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,017::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,018::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,018::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,018::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,018::__init__::225::IOProcess::(_processLogs) (36) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:39,018::__init__::225::IOProcess::(_processLogs) (36) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:39,049::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(36) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,049::__init__::225::IOProcess::(_processLogs) (36) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,049::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >clientIFinit::DEBUG::2014-08-22 09:17:39,049::fileSD::497::Storage.StorageDomain::(activateVolumes) Fixing permissions on /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade >Thread-17::DEBUG::2014-08-22 09:17:39,050::__init__::225::IOProcess::(_processLogs) Sending response sized 666 >Thread-17::DEBUG::2014-08-22 09:17:39,050::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,050::__init__::225::IOProcess::(_processLogs) Message size is 239 >Thread-17::DEBUG::2014-08-22 09:17:39,050::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,051::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >clientIFinit::DEBUG::2014-08-22 09:17:39,051::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /var/run/vdsm/storage/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:17:39,051::__init__::225::IOProcess::(_processLogs) Waiting for next request... >clientIFinit::WARNING::2014-08-22 09:17:39,051::fileUtils::149::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/49f79085-53a0-4d45-8e3e-610bdf644da6 already exists >Thread-17::DEBUG::2014-08-22 09:17:39,052::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >clientIFinit::DEBUG::2014-08-22 09:17:39,052::fileSD::472::Storage.StorageDomain::(createImageLinks) Creating symlink from /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621 to /var/run/vdsm/storage/49f79085-53a0-4d45-8e3e-610bdf644da6/b7b0a6c3-4adf-4a92-9132-00da6ae85621 >clientIFinit::DEBUG::2014-08-22 09:17:39,052::fileSD::477::Storage.StorageDomain::(createImageLinks) img run dir already exists: /var/run/vdsm/storage/49f79085-53a0-4d45-8e3e-610bdf644da6/b7b0a6c3-4adf-4a92-9132-00da6ae85621 >Thread-17::DEBUG::2014-08-22 09:17:39,052::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,053::__init__::225::IOProcess::(_processLogs) (37) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:17:39,053::__init__::225::IOProcess::(_processLogs) (37) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) (37) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) (37) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) Sending response sized 330 >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) Message size is 202 >Thread-17::DEBUG::2014-08-22 09:17:39,054::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) (38) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) (38) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) (38) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,055::__init__::225::IOProcess::(_processLogs) (38) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,056::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,056::__init__::225::IOProcess::(_processLogs) Sending response sized 319 >Thread-17::DEBUG::2014-08-22 09:17:39,056::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,056::__init__::225::IOProcess::(_processLogs) Message size is 215 >Thread-17::DEBUG::2014-08-22 09:17:39,056::__init__::225::IOProcess::(_processLogs) Marshaling message... >clientIFinit::DEBUG::2014-08-22 09:17:39,056::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for f61ef654-8b79-40ad-ba12-f396a913cade >Thread-17::DEBUG::2014-08-22 09:17:39,056::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) (39) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) (39) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) (39) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) (39) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,057::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:39,058::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,058::__init__::225::IOProcess::(_processLogs) Message size is 252 >Thread-17::DEBUG::2014-08-22 09:17:39,058::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,058::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) (40) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) (40) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) (40) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) (40) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:39,059::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,060::__init__::225::IOProcess::(_processLogs) Message size is 257 >Thread-17::DEBUG::2014-08-22 09:17:39,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,060::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >clientIFinit::INFO::2014-08-22 09:17:39,060::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': '49f79085-53a0-4d45-8e3e-610bdf644da6', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'volumeID': u'f61ef654-8b79-40ad-ba12-f396a913cade', 'leasePath': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade.lease', 'imageID': 'b7b0a6c3-4adf-4a92-9132-00da6ae85621'}, 'path': u'/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'imgVolumesInfo': [{'domainID': '49f79085-53a0-4d45-8e3e-610bdf644da6', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'volumeID': u'f61ef654-8b79-40ad-ba12-f396a913cade', 'leasePath': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade.lease', 'imageID': 'b7b0a6c3-4adf-4a92-9132-00da6ae85621'}]} >Thread-17::DEBUG::2014-08-22 09:17:39,060::__init__::225::IOProcess::(_processLogs) Waiting for next request... >clientIFinit::DEBUG::2014-08-22 09:17:39,061::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d809cf6-2f7a-4dd6-8f63-d0caf46495f4`::finished: {'info': {'domainID': '49f79085-53a0-4d45-8e3e-610bdf644da6', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'volumeID': u'f61ef654-8b79-40ad-ba12-f396a913cade', 'leasePath': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade.lease', 'imageID': 'b7b0a6c3-4adf-4a92-9132-00da6ae85621'}, 'path': u'/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'imgVolumesInfo': [{'domainID': '49f79085-53a0-4d45-8e3e-610bdf644da6', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade', 'volumeID': u'f61ef654-8b79-40ad-ba12-f396a913cade', 'leasePath': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade.lease', 'imageID': 'b7b0a6c3-4adf-4a92-9132-00da6ae85621'}]} >Thread-17::DEBUG::2014-08-22 09:17:39,061::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >clientIFinit::DEBUG::2014-08-22 09:17:39,061::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d809cf6-2f7a-4dd6-8f63-d0caf46495f4`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-08-22 09:17:39,061::__init__::225::IOProcess::(_processLogs) Extracting request information... >clientIFinit::DEBUG::2014-08-22 09:17:39,062::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6': < ResourceRef 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6', isValid: 'True' obj: 'None'>} >Thread-17::DEBUG::2014-08-22 09:17:39,062::__init__::225::IOProcess::(_processLogs) (41) Finding callback 'access'... >clientIFinit::DEBUG::2014-08-22 09:17:39,062::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-08-22 09:17:39,062::__init__::225::IOProcess::(_processLogs) (41) Got request for method 'access' >clientIFinit::DEBUG::2014-08-22 09:17:39,063::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' >Thread-17::DEBUG::2014-08-22 09:17:39,063::__init__::225::IOProcess::(_processLogs) (41) Building response >clientIFinit::DEBUG::2014-08-22 09:17:39,063::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' (0 active users) >Thread-17::DEBUG::2014-08-22 09:17:39,064::__init__::225::IOProcess::(_processLogs) (41) Queuing response >clientIFinit::DEBUG::2014-08-22 09:17:39,064::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-08-22 09:17:39,064::__init__::225::IOProcess::(_processLogs) Generating json... >clientIFinit::DEBUG::2014-08-22 09:17:39,064::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6', Clearing records. >Thread-17::DEBUG::2014-08-22 09:17:39,065::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >clientIFinit::DEBUG::2014-08-22 09:17:39,065::task::993::Storage.TaskManager.Task::(_decref) Task=`9d809cf6-2f7a-4dd6-8f63-d0caf46495f4`::ref 0 aborting False >clientIFinit::INFO::2014-08-22 09:17:39,065::clientIF::332::vds::(prepareVolumePath) prepared volume path: /rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/49f79085-53a0-4d45-8e3e-610bdf644da6/images/b7b0a6c3-4adf-4a92-9132-00da6ae85621/f61ef654-8b79-40ad-ba12-f396a913cade >Thread-18::DEBUG::2014-08-22 09:17:39,358::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:17:39,358::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdafcb08-f01c-41e8-986d-8a5f7ad82183`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:39,358::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,359::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,360::__init__::225::IOProcess::(_processLogs) (42) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,360::__init__::225::IOProcess::(_processLogs) (42) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,360::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(42) Building response >Thread-18::DEBUG::2014-08-22 09:17:39,360::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:39,361::__init__::225::IOProcess::(_processLogs) (42) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,361::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-18::INFO::2014-08-22 09:17:39,361::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 6}} >Thread-17::DEBUG::2014-08-22 09:17:39,362::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-18::DEBUG::2014-08-22 09:17:39,362::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdafcb08-f01c-41e8-986d-8a5f7ad82183`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 6}} >Thread-18::DEBUG::2014-08-22 09:17:39,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdafcb08-f01c-41e8-986d-8a5f7ad82183`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:39,362::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:39,362::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:39,362::task::993::Storage.TaskManager.Task::(_decref) Task=`bdafcb08-f01c-41e8-986d-8a5f7ad82183`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:39,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`8cae9a89-adc2-4eb6-97b1-7a5a29a53597`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:39,388::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:17:39,388::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00464446', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020116', 'lastCheck': '3.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:17:39,388::task::1191::Storage.TaskManager.Task::(prepare) Task=`8cae9a89-adc2-4eb6-97b1-7a5a29a53597`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00464446', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020116', 'lastCheck': '3.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:17:39,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`8cae9a89-adc2-4eb6-97b1-7a5a29a53597`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:39,389::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:39,389::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:39,389::task::993::Storage.TaskManager.Task::(_decref) Task=`8cae9a89-adc2-4eb6-97b1-7a5a29a53597`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:39,468::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:39,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:39,469::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-18::ERROR::2014-08-22 09:17:39,469::task::866::Storage.TaskManager.Task::(_setError) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 2163, in getAllTasksStatuses > allTasksStatus = sp.getAllTasksStatuses() > File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper > raise SecureError("Secured object is not in safe state") >SecureError: Secured object is not in safe state >Thread-18::DEBUG::2014-08-22 09:17:39,470::task::885::Storage.TaskManager.Task::(_run) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::Task._run: 649d28ed-bef8-4e31-a9e6-5926cd84475f () {} failed - stopping task >Thread-18::DEBUG::2014-08-22 09:17:39,470::task::1217::Storage.TaskManager.Task::(stop) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::stopping in state preparing (force False) >Thread-18::DEBUG::2014-08-22 09:17:39,470::task::993::Storage.TaskManager.Task::(_decref) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::ref 1 aborting True >Thread-18::INFO::2014-08-22 09:17:39,470::task::1171::Storage.TaskManager.Task::(prepare) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::aborting: Task is aborted: u'Secured object is not in safe state' - code 100 >Thread-18::DEBUG::2014-08-22 09:17:39,470::task::1176::Storage.TaskManager.Task::(prepare) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::Prepare: aborted: Secured object is not in safe state >Thread-18::DEBUG::2014-08-22 09:17:39,470::task::993::Storage.TaskManager.Task::(_decref) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::ref 0 aborting True >Thread-18::DEBUG::2014-08-22 09:17:39,470::task::928::Storage.TaskManager.Task::(_doAbort) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::Task._doAbort: force False >Thread-18::DEBUG::2014-08-22 09:17:39,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:39,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::moving from state preparing -> state aborting >Thread-18::DEBUG::2014-08-22 09:17:39,471::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::_aborting: recover policy none >Thread-18::DEBUG::2014-08-22 09:17:39,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`649d28ed-bef8-4e31-a9e6-5926cd84475f`::moving from state aborting -> state failed >Thread-18::DEBUG::2014-08-22 09:17:39,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:39,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::ERROR::2014-08-22 09:17:39,471::dispatcher::79::Storage.Dispatcher::(wrapper) Secured object is not in safe state >Traceback (most recent call last): > File "/usr/share/vdsm/storage/dispatcher.py", line 71, in wrapper > result = ctask.prepare(func, *args, **kwargs) > File "/usr/share/vdsm/storage/task.py", line 103, in wrapper > return m(self, *a, **kw) > File "/usr/share/vdsm/storage/task.py", line 1179, in prepare > raise self.error >SecureError: Secured object is not in safe state >Thread-18::DEBUG::2014-08-22 09:17:39,583::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:39,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`74ae3b26-16ea-4691-8e47-53fde2b400ad`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:39,584::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:17:39,584::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,584::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,584::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,584::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,584::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,585::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,585::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,585::__init__::225::IOProcess::(_processLogs) (43) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,585::__init__::225::IOProcess::(_processLogs) (43) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,585::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(43) Building response >Thread-18::DEBUG::2014-08-22 09:17:39,585::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:39,586::__init__::225::IOProcess::(_processLogs) (43) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,586::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-18::INFO::2014-08-22 09:17:39,586::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 6}} >Thread-17::DEBUG::2014-08-22 09:17:39,587::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-18::DEBUG::2014-08-22 09:17:39,587::task::1191::Storage.TaskManager.Task::(prepare) Task=`74ae3b26-16ea-4691-8e47-53fde2b400ad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 6}} >Thread-18::DEBUG::2014-08-22 09:17:39,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`74ae3b26-16ea-4691-8e47-53fde2b400ad`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:39,587::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:39,587::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:39,587::task::993::Storage.TaskManager.Task::(_decref) Task=`74ae3b26-16ea-4691-8e47-53fde2b400ad`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:39,595::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:39,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac3e9d48-206e-4894-9002-e61acff1258f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:39,595::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:17:39,595::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,595::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,595::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) (44) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) (44) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,596::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(44) Building response >Thread-18::DEBUG::2014-08-22 09:17:39,596::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:39,597::__init__::225::IOProcess::(_processLogs) (44) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,597::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-18::INFO::2014-08-22 09:17:39,598::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 6}} >Thread-17::DEBUG::2014-08-22 09:17:39,598::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-18::DEBUG::2014-08-22 09:17:39,598::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac3e9d48-206e-4894-9002-e61acff1258f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 6}} >Thread-18::DEBUG::2014-08-22 09:17:39,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac3e9d48-206e-4894-9002-e61acff1258f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:39,599::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:39,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:39,599::task::993::Storage.TaskManager.Task::(_decref) Task=`ac3e9d48-206e-4894-9002-e61acff1258f`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:39,700::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:39,701::task::595::Storage.TaskManager.Task::(_updateState) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:39,701::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', prevID=2, prevLVER='6', maxHostID=250, domVersion='3', options=None) >Thread-18::DEBUG::2014-08-22 09:17:39,701::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`22856ff2-f482-46db-86b9-1eb226ee1687`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '572' at 'spmStart' >Thread-18::DEBUG::2014-08-22 09:17:39,701::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'exclusive' >Thread-18::DEBUG::2014-08-22 09:17:39,701::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'exclusive' (1 active user) >Thread-18::DEBUG::2014-08-22 09:17:39,701::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`22856ff2-f482-46db-86b9-1eb226ee1687`::Granted request >Thread-18::DEBUG::2014-08-22 09:17:39,702::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (exclusive) >Thread-18::DEBUG::2014-08-22 09:17:39,702::task::993::Storage.TaskManager.Task::(_decref) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::ref 1 aborting False >Thread-18::DEBUG::2014-08-22 09:17:39,702::taskManager::78::Storage.TaskManager::(scheduleJob) scheduled job spmStart for task 0887d42f-13ba-43ba-98a2-0a36e297693d >Thread-18::INFO::2014-08-22 09:17:39,702::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-18::DEBUG::2014-08-22 09:17:39,702::task::1183::Storage.TaskManager.Task::(prepare) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::Prepare: 1 jobs exist, move to acquiring >Thread-18::DEBUG::2014-08-22 09:17:39,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::moving from state preparing -> state acquiring >Thread-18::DEBUG::2014-08-22 09:17:39,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::moving from state acquiring -> state queued >Thread-18::DEBUG::2014-08-22 09:17:39,702::taskManager::56::Storage.TaskManager::(_queueTask) queuing task: 0887d42f-13ba-43ba-98a2-0a36e297693d >Thread-18::DEBUG::2014-08-22 09:17:39,702::taskManager::64::Storage.TaskManager::(_queueTask) task queued: 0887d42f-13ba-43ba-98a2-0a36e297693d >Thread-1::DEBUG::2014-08-22 09:17:39,702::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-18::DEBUG::2014-08-22 09:17:39,703::task::1188::Storage.TaskManager.Task::(prepare) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::returning >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,703::threadPool::200::Storage.ThreadPool.WorkerThread::(_processNextTask) Task: 0887d42f-13ba-43ba-98a2-0a36e297693d running: <bound method Task.commit of <storage.task.Task instance at 0x7f1f381c9518>> with: None >Thread-18::DEBUG::2014-08-22 09:17:39,704::task::993::Storage.TaskManager.Task::(_decref) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::ref 0 aborting False >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,704::task::1200::Storage.TaskManager.Task::(commit) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::committing task: 0887d42f-13ba-43ba-98a2-0a36e297693d >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::moving from state queued -> state running >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,705::task::905::Storage.TaskManager.Task::(_runJobs) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f1f0c3b5210>> (args: (2, '6', 250, 3) kwargs: {}) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,706::task::331::Storage.TaskManager.Task::(run) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f1f0c3b5210>> (args: (2, '6', 250, 3) kwargs: {}) callback None >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,706::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,706::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-17::DEBUG::2014-08-22 09:17:39,706::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) (45) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) (45) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,707::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(45) Building response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,708::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:39,708::__init__::225::IOProcess::(_processLogs) (45) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,709::__init__::225::IOProcess::(_processLogs) Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,709::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-17::DEBUG::2014-08-22 09:17:39,709::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,710::__init__::225::IOProcess::(_processLogs) (46) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,711::__init__::225::IOProcess::(_processLogs) (46) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,711::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(46) Building response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,711::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:39,711::__init__::225::IOProcess::(_processLogs) (46) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,712::__init__::225::IOProcess::(_processLogs) Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::INFO::2014-08-22 09:17:39,712::clusterlock::183::Storage.SANLock::(acquireHostId) Acquiring host id for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 (id: 2) >Thread-17::DEBUG::2014-08-22 09:17:39,712::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,713::clusterlock::201::Storage.SANLock::(acquireHostId) Host id for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 successfully acquired (id: 2) >0887d42f-13ba-43ba-98a2-0a36e297693d::INFO::2014-08-22 09:17:39,713::clusterlock::234::Storage.SANLock::(acquire) Acquiring cluster lock for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 (id: 2) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,792::clusterlock::260::Storage.SANLock::(acquire) Cluster lock for domain 49f79085-53a0-4d45-8e3e-610bdf644da6 successfully acquired (id: 2) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,792::sp::272::Storage.StoragePool::(startSpm) spm lock acquired successfully >Thread-17::DEBUG::2014-08-22 09:17:39,792::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,792::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,792::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,792::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,792::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,792::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,793::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,793::__init__::225::IOProcess::(_processLogs) (47) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,793::__init__::225::IOProcess::(_processLogs) (47) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,797::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(47) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,797::__init__::225::IOProcess::(_processLogs) (47) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,797::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,797::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=6', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=5e983bcd040c7dc451c7ea59e7f9a13114e6fd2a'] >Thread-17::DEBUG::2014-08-22 09:17:39,798::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,799::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,799::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,799::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) Message size is 972 >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) (48) Finding callback 'writefile'... >Thread-17::DEBUG::2014-08-22 09:17:39,800::__init__::225::IOProcess::(_processLogs) (48) Got request for method 'writefile' >Thread-17::DEBUG::2014-08-22 09:17:39,853::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(48) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,854::__init__::225::IOProcess::(_processLogs) (48) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,854::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::WARNING::2014-08-22 09:17:39,854::outOfProcess::221::oop::(rename) renaming a directory is not an atomic operation >Thread-17::DEBUG::2014-08-22 09:17:39,855::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:17:39,855::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,855::__init__::225::IOProcess::(_processLogs) Message size is 318 >Thread-17::DEBUG::2014-08-22 09:17:39,855::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,856::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,856::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,856::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,856::__init__::225::IOProcess::(_processLogs) (49) Finding callback 'rename'... >Thread-17::DEBUG::2014-08-22 09:17:39,856::__init__::225::IOProcess::(_processLogs) (49) Got request for method 'rename' >Thread-17::DEBUG::2014-08-22 09:17:39,865::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(49) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,865::__init__::225::IOProcess::(_processLogs) (49) Queuing response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,865::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-17::DEBUG::2014-08-22 09:17:39,866::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,866::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,867::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f374ff41-1402-42bc-b835-61a4774e826d`::Request was made in '/usr/share/vdsm/storage/sp.py' line '426' at '_upgradePool' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,867::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'exclusive' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,867::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'exclusive' (1 active user) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,867::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f374ff41-1402-42bc-b835-61a4774e826d`::Granted request >0887d42f-13ba-43ba-98a2-0a36e297693d::INFO::2014-08-22 09:17:39,868::sp::432::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `49f79085-53a0-4d45-8e3e-610bdf644da6` >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,868::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.49f79085-53a0-4d45-8e3e-610bdf644da6`ReqID=`d4d92b9a-3aae-406a-8098-54706c52246d`::Request was made in '/usr/share/vdsm/storage/sp.py' line '434' at '_upgradePool' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,868::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' for lock type 'exclusive' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,868::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' is free. Now locking as 'exclusive' (1 active user) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,868::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.49f79085-53a0-4d45-8e3e-610bdf644da6`ReqID=`d4d92b9a-3aae-406a-8098-54706c52246d`::Granted request >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) Message size is 139 >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) (50) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) (50) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:39,870::__init__::225::IOProcess::(_processLogs) (50) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,871::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(50) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Sending response sized 189 >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) (51) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) (51) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:17:39,872::__init__::225::IOProcess::(_processLogs) (51) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,873::__init__::225::IOProcess::(_processLogs) (51) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,873::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,873::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,875::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,875::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,876::__init__::225::IOProcess::(_processLogs) (52) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:17:39,876::__init__::225::IOProcess::(_processLogs) (52) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:17:39,887::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(52) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,887::__init__::225::IOProcess::(_processLogs) (52) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,888::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,888::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:17:39,889::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:39,889::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,889::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,889::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) (53) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) (53) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,890::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(53) Building response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,890::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:17:39,891::__init__::225::IOProcess::(_processLogs) (53) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,892::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,893::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-17::DEBUG::2014-08-22 09:17:39,893::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,893::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 09:17:39,893::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,893::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) (54) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) (54) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) (54) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) (54) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,894::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:39,895::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,895::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 09:17:39,895::__init__::225::IOProcess::(_processLogs) Marshaling message... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,895::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:17:39,895::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >0887d42f-13ba-43ba-98a2-0a36e297693d::INFO::2014-08-22 09:17:39,896::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:17:39,896::__init__::225::IOProcess::(_processLogs) Waiting for next request... >0887d42f-13ba-43ba-98a2-0a36e297693d::INFO::2014-08-22 09:17:39,896::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:17:39,897::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,898::__init__::225::IOProcess::(_processLogs) Extracting request information... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,898::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' >Thread-17::DEBUG::2014-08-22 09:17:39,898::__init__::225::IOProcess::(_processLogs) (55) Finding callback 'glob'... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,899::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' (0 active users) >Thread-17::DEBUG::2014-08-22 09:17:39,899::__init__::225::IOProcess::(_processLogs) (55) Got request for method 'glob' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,899::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-08-22 09:17:39,900::__init__::225::IOProcess::(_processLogs) (55) Building response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,900::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.49f79085-53a0-4d45-8e3e-610bdf644da6', Clearing records. >Thread-17::DEBUG::2014-08-22 09:17:39,900::__init__::225::IOProcess::(_processLogs) (55) Queuing response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,901::sp::437::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >Thread-17::DEBUG::2014-08-22 09:17:39,901::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,902::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:17:39,902::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,902::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) (56) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) (56) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,903::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|(56) Building response >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,904::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:17:39,904::__init__::225::IOProcess::(_processLogs) (56) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,905::__init__::225::IOProcess::(_processLogs) Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,905::sp::444::Storage.StoragePool::(_upgradePool) Registering with state change event >Thread-17::DEBUG::2014-08-22 09:17:39,906::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,906::sp::447::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >Thread-39::DEBUG::2014-08-22 09:17:39,907::sp::152::Storage.StoragePool::(_upgradePoolDomain) Preparing to upgrade domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,907::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,908::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-39::DEBUG::2014-08-22 09:17:39,908::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07`ReqID=`cb4c68fe-c99a-4e23-ba2b-edb22e193268`::Request was made in '/usr/share/vdsm/storage/sp.py' line '163' at '_upgradePoolDomain' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,908::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-39::DEBUG::2014-08-22 09:17:39,908::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07' for lock type 'exclusive' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,909::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-39::DEBUG::2014-08-22 09:17:39,909::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07' is free. Now locking as 'exclusive' (1 active user) >Thread-39::DEBUG::2014-08-22 09:17:39,909::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07`ReqID=`cb4c68fe-c99a-4e23-ba2b-edb22e193268`::Granted request >Thread-17::DEBUG::2014-08-22 09:17:39,910::__init__::225::IOProcess::(_processLogs) (29) Got request for method 'glob'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,911::__init__::225::IOProcess::(_processLogs) Message size is 178 >Thread-17::DEBUG::2014-08-22 09:17:39,911::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-39::DEBUG::2014-08-22 09:17:39,911::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.7c4cb940-6043-48fa-88f8-ab4762019c07`ReqID=`b0cd5131-3694-48d4-a9e6-6b22cc540770`::Request was made in '/usr/share/vdsm/storage/sp.py' line '165' at '_upgradePoolDomain' >Thread-17::DEBUG::2014-08-22 09:17:39,912::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-39::DEBUG::2014-08-22 09:17:39,912::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' for lock type 'exclusive' >Thread-17::DEBUG::2014-08-22 09:17:39,912::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-39::DEBUG::2014-08-22 09:17:39,913::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' is free. Now locking as 'exclusive' (1 active user) >Thread-17::DEBUG::2014-08-22 09:17:39,913::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-39::DEBUG::2014-08-22 09:17:39,913::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.7c4cb940-6043-48fa-88f8-ab4762019c07`ReqID=`b0cd5131-3694-48d4-a9e6-6b22cc540770`::Granted request >Thread-17::DEBUG::2014-08-22 09:17:39,913::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-39::DEBUG::2014-08-22 09:17:39,915::sp::175::Storage.StoragePool::(_upgradePoolDomain) Domain `7c4cb940-6043-48fa-88f8-ab4762019c07` is not a data domain it is an 2 domain, not upgrading >Thread-17::DEBUG::2014-08-22 09:17:39,915::__init__::225::IOProcess::(_processLogs) (57) Finding callback 'access'... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,915::sd::644::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/master/vms >Thread-39::DEBUG::2014-08-22 09:17:39,915::sp::192::Storage.StoragePool::(_upgradePoolDomain) All domains are upgraded, unregistering from state change event >Thread-17::DEBUG::2014-08-22 09:17:39,916::__init__::225::IOProcess::(_processLogs) (57) Got request for method 'access' >Thread-39::DEBUG::2014-08-22 09:17:39,916::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' >Thread-17::DEBUG::2014-08-22 09:17:39,917::__init__::225::IOProcess::(_processLogs) (57) Building response >Thread-39::DEBUG::2014-08-22 09:17:39,917::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' (0 active users) >Thread-17::DEBUG::2014-08-22 09:17:39,917::__init__::225::IOProcess::(_processLogs) (57) Queuing response >Thread-39::DEBUG::2014-08-22 09:17:39,917::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-08-22 09:17:39,918::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-39::DEBUG::2014-08-22 09:17:39,918::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07', Clearing records. >Thread-17::DEBUG::2014-08-22 09:17:39,918::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-39::DEBUG::2014-08-22 09:17:39,919::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07' >Thread-17::DEBUG::2014-08-22 09:17:39,919::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-39::DEBUG::2014-08-22 09:17:39,919::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07' (0 active users) >Thread-39::DEBUG::2014-08-22 09:17:39,920::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-08-22 09:17:39,920::__init__::225::IOProcess::(_processLogs) Message size is 69 >Thread-39::DEBUG::2014-08-22 09:17:39,920::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_7c4cb940-6043-48fa-88f8-ab4762019c07', Clearing records. >Thread-17::DEBUG::2014-08-22 09:17:39,920::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,921::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,921::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,921::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,921::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,921::__init__::225::IOProcess::(_processLogs) (58) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,921::__init__::225::IOProcess::(_processLogs) (58) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) (58) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) (58) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) Message size is 73 >Thread-17::DEBUG::2014-08-22 09:17:39,922::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) (59) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) (59) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) (59) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,923::__init__::225::IOProcess::(_processLogs) (59) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,924::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,924::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,924::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,924::__init__::225::IOProcess::(_processLogs) Message size is 85 >Thread-17::DEBUG::2014-08-22 09:17:39,924::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,924::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) (60) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) (60) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) (60) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) (60) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,925::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,926::__init__::225::IOProcess::(_processLogs) Message size is 89 >Thread-17::DEBUG::2014-08-22 09:17:39,926::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,926::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,926::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,926::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,926::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) (61) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) (61) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) (61) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) (61) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) Message size is 135 >Thread-17::DEBUG::2014-08-22 09:17:39,927::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) (62) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) (62) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) (62) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) (62) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,928::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,929::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,929::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,929::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:17:39,929::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,929::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) (63) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) (63) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) (63) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) (63) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,930::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,931::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:17:39,931::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,931::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,931::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,931::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,931::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) (64) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) (64) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) (64) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) (64) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:17:39,932::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,933::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,933::sd::647::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/master/tasks >Thread-17::DEBUG::2014-08-22 09:17:39,933::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) (65) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) (65) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) (65) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) (65) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,934::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,935::__init__::225::IOProcess::(_processLogs) Message size is 69 >Thread-17::DEBUG::2014-08-22 09:17:39,935::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,935::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,935::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,935::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) (66) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) (66) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) (66) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) (66) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) Message size is 73 >Thread-17::DEBUG::2014-08-22 09:17:39,936::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) (67) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) (67) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,937::__init__::225::IOProcess::(_processLogs) (67) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) (67) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) Message size is 85 >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,938::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) (68) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) (68) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) (68) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) (68) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,939::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,940::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,940::__init__::225::IOProcess::(_processLogs) Message size is 89 >Thread-17::DEBUG::2014-08-22 09:17:39,940::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,940::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,940::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,940::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) (69) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) (69) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) (69) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) (69) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) Message size is 135 >Thread-17::DEBUG::2014-08-22 09:17:39,941::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) (70) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) (70) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,942::__init__::225::IOProcess::(_processLogs) (70) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) (70) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,943::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) (71) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) (71) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) (71) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) (71) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,944::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) (72) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,945::__init__::225::IOProcess::(_processLogs) (72) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,946::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(72) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,946::__init__::225::IOProcess::(_processLogs) (72) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,946::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,946::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,946::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,946::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) (73) Finding callback 'mkdir'... >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) (73) Got request for method 'mkdir' >Thread-17::DEBUG::2014-08-22 09:17:39,947::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(73) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,948::__init__::225::IOProcess::(_processLogs) (73) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,948::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,948::__init__::225::IOProcess::(_processLogs) Sending response sized 78 >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) (74) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:17:39,949::__init__::225::IOProcess::(_processLogs) (74) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:17:39,950::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(74) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,950::__init__::225::IOProcess::(_processLogs) (74) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,950::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,951::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:17:39,951::__init__::225::IOProcess::(_processLogs) Sending response sized 830 >Thread-17::DEBUG::2014-08-22 09:17:39,953::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,953::__init__::225::IOProcess::(_processLogs) Message size is 171 >Thread-17::DEBUG::2014-08-22 09:17:39,953::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) (75) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) (75) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(75) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,954::__init__::225::IOProcess::(_processLogs) (75) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,955::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,955::__init__::225::IOProcess::(_processLogs) Sending response sized 324 >Thread-17::DEBUG::2014-08-22 09:17:39,955::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:39,955::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:17:39,955::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:39,956::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,956::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-17::DEBUG::2014-08-22 09:17:39,956::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:39,957::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:39,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:39,959::__init__::225::IOProcess::(_processLogs) (76) Finding callback 'stat'... >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,960::storage_mailbox::574::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1'] >Thread-17::DEBUG::2014-08-22 09:17:39,960::__init__::225::IOProcess::(_processLogs) (76) Got request for method 'stat' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,960::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) dd of=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:39,961::__init__::225::IOProcess::(_processLogs) (76) Building response >Thread-17::DEBUG::2014-08-22 09:17:39,966::__init__::225::IOProcess::(_processLogs) (76) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:39,967::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:39,967::__init__::225::IOProcess::(_processLogs) Sending response sized 326 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,996::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305571 s, 33.5 MB/s\n'; <rc> = 0 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,996::storage_mailbox::582::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool da0e21b4-cba8-40bf-a5cd-3a50c2199246 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,997::sp::319::Storage.StoragePool::(startSpm) SPM mailbox ready for pool da0e21b4-cba8-40bf-a5cd-3a50c2199246 on master domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,997::sp::327::Storage.StoragePool::(startSpm) ended. >Dummy-45::DEBUG::2014-08-22 09:17:39,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:39,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::moving from state running -> state finished >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,003::task::916::Storage.TaskManager.Task::(_runJobs) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::Task.run: exit - success: result >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,004::task::993::Storage.TaskManager.Task::(_decref) Task=`0887d42f-13ba-43ba-98a2-0a36e297693d`::ref 0 aborting False >0887d42f-13ba-43ba-98a2-0a36e297693d::DEBUG::2014-08-22 09:17:40,004::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 0 >Dummy-45::DEBUG::2014-08-22 09:17:40,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122968 s, 83.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:40,713::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:40,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`feff9680-6a6f-40de-8740-4ca2a1c2f688`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:40,713::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='0887d42f-13ba-43ba-98a2-0a36e297693d', spUUID=None, options=None) >Thread-18::DEBUG::2014-08-22 09:17:40,713::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 0887d42f-13ba-43ba-98a2-0a36e297693d >Thread-18::DEBUG::2014-08-22 09:17:40,714::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '0887d42f-13ba-43ba-98a2-0a36e297693d'} >Thread-18::INFO::2014-08-22 09:17:40,714::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '0887d42f-13ba-43ba-98a2-0a36e297693d'}} >Thread-18::DEBUG::2014-08-22 09:17:40,714::task::1191::Storage.TaskManager.Task::(prepare) Task=`feff9680-6a6f-40de-8740-4ca2a1c2f688`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '0887d42f-13ba-43ba-98a2-0a36e297693d'}} >Thread-18::DEBUG::2014-08-22 09:17:40,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`feff9680-6a6f-40de-8740-4ca2a1c2f688`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:40,714::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:40,714::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:40,714::task::993::Storage.TaskManager.Task::(_decref) Task=`feff9680-6a6f-40de-8740-4ca2a1c2f688`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:40,719::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:40,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`d79ed8c9-b6a5-4ff6-9bf6-37b11b5282f9`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:40,719::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:17:40,719::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:17:40,719::task::1191::Storage.TaskManager.Task::(prepare) Task=`d79ed8c9-b6a5-4ff6-9bf6-37b11b5282f9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:17:40,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`d79ed8c9-b6a5-4ff6-9bf6-37b11b5282f9`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:40,720::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:40,720::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:40,720::task::993::Storage.TaskManager.Task::(_decref) Task=`d79ed8c9-b6a5-4ff6-9bf6-37b11b5282f9`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:40,768::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:40,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`f88cefcf-848e-4ee6-b5a6-611af67b75fe`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:40,769::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='0887d42f-13ba-43ba-98a2-0a36e297693d', spUUID=None, options=None) >Thread-18::DEBUG::2014-08-22 09:17:40,769::taskManager::171::Storage.TaskManager::(clearTask) Entry. taskID: 0887d42f-13ba-43ba-98a2-0a36e297693d >Thread-18::DEBUG::2014-08-22 09:17:40,769::taskManager::176::Storage.TaskManager::(clearTask) Return. >Thread-18::INFO::2014-08-22 09:17:40,769::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-18::DEBUG::2014-08-22 09:17:40,769::task::1191::Storage.TaskManager.Task::(prepare) Task=`f88cefcf-848e-4ee6-b5a6-611af67b75fe`::finished: None >Thread-18::DEBUG::2014-08-22 09:17:40,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`f88cefcf-848e-4ee6-b5a6-611af67b75fe`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:40,769::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:40,769::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:40,769::task::993::Storage.TaskManager.Task::(_decref) Task=`f88cefcf-848e-4ee6-b5a6-611af67b75fe`::ref 0 aborting False >Detector thread::DEBUG::2014-08-22 09:17:40,876::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.60:59452 >Detector thread::DEBUG::2014-08-22 09:17:40,876::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.60:59452 >Detector thread::DEBUG::2014-08-22 09:17:40,876::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.34.63.60:59452 >Detector thread::DEBUG::2014-08-22 09:17:40,876::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.34.63.60', 59452) >Thread-46::DEBUG::2014-08-22 09:17:40,877::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-46::DEBUG::2014-08-22 09:17:40,878::task::595::Storage.TaskManager.Task::(_updateState) Task=`f19f8fa7-a9e3-4992-af46-28bed616b8de`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:17:40,878::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:17:40,878::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`732ac787-f82f-496c-96d6-b8930c4a6d8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:17:40,878::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:17:40,878::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:17:40,879::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`732ac787-f82f-496c-96d6-b8930c4a6d8a`::Granted request >Thread-46::DEBUG::2014-08-22 09:17:40,879::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f19f8fa7-a9e3-4992-af46-28bed616b8de`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:17:40,879::task::993::Storage.TaskManager.Task::(_decref) Task=`f19f8fa7-a9e3-4992-af46-28bed616b8de`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:17:40,881::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:17:40,881::task::1191::Storage.TaskManager.Task::(prepare) Task=`f19f8fa7-a9e3-4992-af46-28bed616b8de`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:17:40,881::task::595::Storage.TaskManager.Task::(_updateState) Task=`f19f8fa7-a9e3-4992-af46-28bed616b8de`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:17:40,881::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:17:40,881::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:17:40,881::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:17:40,881::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:17:40,882::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:17:40,882::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:17:40,882::task::993::Storage.TaskManager.Task::(_decref) Task=`f19f8fa7-a9e3-4992-af46-28bed616b8de`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:17:40,953::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-18::DEBUG::2014-08-22 09:17:40,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`1240f3f4-4cfa-43c3-bee2-f859bb729585`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:40,954::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksInfo(spUUID=None, options=None) >Thread-18::DEBUG::2014-08-22 09:17:40,954::taskManager::191::Storage.TaskManager::(getAllTasksInfo) Entry. >Thread-18::DEBUG::2014-08-22 09:17:40,954::taskManager::202::Storage.TaskManager::(getAllTasksInfo) Return. Response: {} >Thread-18::INFO::2014-08-22 09:17:40,954::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksInfo, Return response: {'allTasksInfo': {}} >Thread-18::DEBUG::2014-08-22 09:17:40,954::task::1191::Storage.TaskManager.Task::(prepare) Task=`1240f3f4-4cfa-43c3-bee2-f859bb729585`::finished: {'allTasksInfo': {}} >Thread-18::DEBUG::2014-08-22 09:17:40,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`1240f3f4-4cfa-43c3-bee2-f859bb729585`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:40,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:40,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:40,954::task::993::Storage.TaskManager.Task::(_decref) Task=`1240f3f4-4cfa-43c3-bee2-f859bb729585`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:17:42,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:42,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011602 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:17:44,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:44,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121616 s, 84.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:45,881::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:45,881::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:17:45,881::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,882::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:45,882::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,882::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,882::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:45,883::__init__::225::IOProcess::(_processLogs) (77) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:45,883::__init__::225::IOProcess::(_processLogs) (77) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:45,883::__init__::225::IOProcess::(_processLogs) (77) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,883::__init__::225::IOProcess::(_processLogs) (77) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,884::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:17:45,884::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:45,885::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-22::DEBUG::2014-08-22 09:17:45,895::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000264488 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:45,896::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:45,896::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:17:45,896::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,896::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) (78) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) (78) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) (78) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,897::__init__::225::IOProcess::(_processLogs) (78) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,898::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:17:45,899::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-17::DEBUG::2014-08-22 09:17:45,899::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:45,899::__init__::225::IOProcess::(_processLogs) Message size is 178 >Thread-17::DEBUG::2014-08-22 09:17:45,899::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,900::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:45,900::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,900::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,900::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:45,900::__init__::225::IOProcess::(_processLogs) (79) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:45,900::__init__::225::IOProcess::(_processLogs) (79) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) (79) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) (79) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:17:45,901::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,902::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:45,902::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,902::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,902::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) (80) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) (80) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) (80) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) (80) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:17:45,903::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,904::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:45,904::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) (81) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) (81) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) (81) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) (81) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:45,905::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:45,937::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:45,938::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:45,940::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,941::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-21::DEBUG::2014-08-22 09:17:45,941::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:45,942::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,951::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,952::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:17:45,954::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000233811 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:45,954::__init__::225::IOProcess::(_processLogs) (82) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:45,956::__init__::225::IOProcess::(_processLogs) (82) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:45,956::__init__::225::IOProcess::(_processLogs) (82) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,956::__init__::225::IOProcess::(_processLogs) (82) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,956::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:45,957::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-17::DEBUG::2014-08-22 09:17:45,957::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:45,958::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:45,959::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:45,960::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:45,960::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:45,960::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:45,960::__init__::225::IOProcess::(_processLogs) (83) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:45,961::__init__::225::IOProcess::(_processLogs) (83) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:45,961::__init__::225::IOProcess::(_processLogs) (83) Building response >Thread-17::DEBUG::2014-08-22 09:17:45,961::__init__::225::IOProcess::(_processLogs) (83) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:45,961::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:45,963::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 327 >Dummy-45::DEBUG::2014-08-22 09:17:46,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:46,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117724 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:17:48,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:48,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117977 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:17:50,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:50,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117565 s, 87.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:50,953::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-18::DEBUG::2014-08-22 09:17:50,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd7818ed-0271-4849-ac1a-615ff6f5e500`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:50,953::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:17:50,954::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:17:50,954::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd7818ed-0271-4849-ac1a-615ff6f5e500`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:17:50,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd7818ed-0271-4849-ac1a-615ff6f5e500`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:50,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:50,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:50,954::task::993::Storage.TaskManager.Task::(_decref) Task=`cd7818ed-0271-4849-ac1a-615ff6f5e500`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:17:51,043::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-46::DEBUG::2014-08-22 09:17:51,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f058820-f26b-4f9f-aaab-23d7265555ac`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:17:51,043::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:17:51,043::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2b40b016-ec5d-434b-9af7-e5da968281d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:17:51,044::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:17:51,044::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:17:51,044::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2b40b016-ec5d-434b-9af7-e5da968281d1`::Granted request >Thread-46::DEBUG::2014-08-22 09:17:51,044::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8f058820-f26b-4f9f-aaab-23d7265555ac`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:17:51,044::task::993::Storage.TaskManager.Task::(_decref) Task=`8f058820-f26b-4f9f-aaab-23d7265555ac`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:17:51,046::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:17:51,046::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f058820-f26b-4f9f-aaab-23d7265555ac`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:17:51,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f058820-f26b-4f9f-aaab-23d7265555ac`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:17:51,047::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:17:51,047::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:17:51,047::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:17:51,047::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:17:51,047::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:17:51,047::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:17:51,047::task::993::Storage.TaskManager.Task::(_decref) Task=`8f058820-f26b-4f9f-aaab-23d7265555ac`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:17:52,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:52,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117232 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:17:54,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:54,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116447 s, 87.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:17:54,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`192c9aa2-0fec-472c-99b0-fe963dd2d7b5`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:17:54,759::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:17:54,759::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233811', 'lastCheck': '8.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264488', 'lastCheck': '8.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:17:54,759::task::1191::Storage.TaskManager.Task::(prepare) Task=`192c9aa2-0fec-472c-99b0-fe963dd2d7b5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233811', 'lastCheck': '8.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264488', 'lastCheck': '8.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:17:54,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`192c9aa2-0fec-472c-99b0-fe963dd2d7b5`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:17:54,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:17:54,760::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:17:54,760::task::993::Storage.TaskManager.Task::(_decref) Task=`192c9aa2-0fec-472c-99b0-fe963dd2d7b5`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:17:55,905::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,905::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:17:55,905::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,905::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,905::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,906::__init__::225::IOProcess::(_processLogs) (84) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:55,906::__init__::225::IOProcess::(_processLogs) (84) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:55,906::__init__::225::IOProcess::(_processLogs) DEBUG|(84) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,907::__init__::225::IOProcess::(_processLogs) (84) Queuing response >Thread-22::DEBUG::2014-08-22 09:17:55,907::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:17:55,907::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,913::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-22::DEBUG::2014-08-22 09:17:55,913::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000229461 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) (85) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:55,914::__init__::225::IOProcess::(_processLogs) (85) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:55,915::__init__::225::IOProcess::(_processLogs) DEBUG|(85) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,916::__init__::225::IOProcess::(_processLogs) (85) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:55,916::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,916::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-17::DEBUG::2014-08-22 09:17:55,916::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,916::__init__::225::IOProcess::(_processLogs) Message size is 178 >Thread-17::DEBUG::2014-08-22 09:17:55,916::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) (86) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) (86) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:55,917::__init__::225::IOProcess::(_processLogs) (86) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) (86) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,918::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,919::__init__::225::IOProcess::(_processLogs) (87) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:55,919::__init__::225::IOProcess::(_processLogs) (87) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:55,919::__init__::225::IOProcess::(_processLogs) (87) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,919::__init__::225::IOProcess::(_processLogs) (87) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:55,919::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,919::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) (88) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) (88) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:17:55,920::__init__::225::IOProcess::(_processLogs) (88) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,921::__init__::225::IOProcess::(_processLogs) (88) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:55,921::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,921::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,965::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,966::__init__::225::IOProcess::(_processLogs) (89) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:55,966::__init__::225::IOProcess::(_processLogs) (89) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:55,966::__init__::225::IOProcess::(_processLogs) (89) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,966::__init__::225::IOProcess::(_processLogs) (89) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:55,966::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,966::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 327 >Thread-21::DEBUG::2014-08-22 09:17:55,967::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:17:55,973::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000303716 s, 1.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) (90) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:17:55,974::__init__::225::IOProcess::(_processLogs) (90) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:17:55,975::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(90) Building response >Thread-17::DEBUG::2014-08-22 09:17:55,976::__init__::225::IOProcess::(_processLogs) (90) Queuing response >Thread-17::DEBUG::2014-08-22 09:17:55,976::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:17:55,976::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Dummy-45::DEBUG::2014-08-22 09:17:56,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:56,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120155 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:17:58,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:17:58,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117264 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:00,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:00,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118879 s, 86.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:01,097::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:18:01,097::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ad4ed47-d7eb-4faf-8f89-acb84feffb0c`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:01,098::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:18:01,098::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:01,098::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ad4ed47-d7eb-4faf-8f89-acb84feffb0c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:01,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ad4ed47-d7eb-4faf-8f89-acb84feffb0c`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:01,098::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:01,098::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:01,098::task::993::Storage.TaskManager.Task::(_decref) Task=`2ad4ed47-d7eb-4faf-8f89-acb84feffb0c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:18:01,141::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:18:01,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7dd7492-9c39-4572-aeca-d4d53d2a8d91`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:18:01,142::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:18:01,142::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`de761877-5d52-45f0-bbbc-1cb8b6bb3d2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:18:01,142::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:18:01,142::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:18:01,142::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`de761877-5d52-45f0-bbbc-1cb8b6bb3d2e`::Granted request >Thread-46::DEBUG::2014-08-22 09:18:01,143::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b7dd7492-9c39-4572-aeca-d4d53d2a8d91`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:18:01,143::task::993::Storage.TaskManager.Task::(_decref) Task=`b7dd7492-9c39-4572-aeca-d4d53d2a8d91`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:18:01,145::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:01,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7dd7492-9c39-4572-aeca-d4d53d2a8d91`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:01,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7dd7492-9c39-4572-aeca-d4d53d2a8d91`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:18:01,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:18:01,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:18:01,145::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:18:01,145::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:18:01,145::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:18:01,145::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:18:01,146::task::993::Storage.TaskManager.Task::(_decref) Task=`b7dd7492-9c39-4572-aeca-d4d53d2a8d91`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:02,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:02,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217441 s, 47.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:04,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:04,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114932 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,922::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,923::__init__::225::IOProcess::(_processLogs) (91) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:05,923::__init__::225::IOProcess::(_processLogs) (91) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:05,923::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(91) Building response >Thread-22::DEBUG::2014-08-22 09:18:05,924::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:05,924::__init__::225::IOProcess::(_processLogs) (91) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,929::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,930::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-22::DEBUG::2014-08-22 09:18:05,930::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000271423 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:05,930::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) (92) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:05,931::__init__::225::IOProcess::(_processLogs) (92) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:05,932::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(92) Building response >Thread-17::DEBUG::2014-08-22 09:18:05,932::__init__::225::IOProcess::(_processLogs) (92) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,933::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,933::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-17::DEBUG::2014-08-22 09:18:05,933::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,933::__init__::225::IOProcess::(_processLogs) Message size is 178 >Thread-17::DEBUG::2014-08-22 09:18:05,933::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,933::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) (93) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) (93) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) (93) Building response >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) (93) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,934::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:18:05,935::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,935::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:18:05,935::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,936::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,936::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:05,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,936::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,936::__init__::225::IOProcess::(_processLogs) (94) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:05,936::__init__::225::IOProcess::(_processLogs) (94) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) (94) Building response >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) (94) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:18:05,937::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) (95) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) (95) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:05,938::__init__::225::IOProcess::(_processLogs) (95) Building response >Thread-17::DEBUG::2014-08-22 09:18:05,939::__init__::225::IOProcess::(_processLogs) (95) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,939::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,939::__init__::225::IOProcess::(_processLogs) Sending response sized 72 >Thread-17::DEBUG::2014-08-22 09:18:05,976::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,976::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:18:05,976::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,977::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,978::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:18:05,978::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:05,979::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,984::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,984::__init__::225::IOProcess::(_processLogs) (96) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:05,984::__init__::225::IOProcess::(_processLogs) (96) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:05,984::__init__::225::IOProcess::(_processLogs) (96) Building response >Thread-17::DEBUG::2014-08-22 09:18:05,984::__init__::225::IOProcess::(_processLogs) (96) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,984::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,985::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-21::DEBUG::2014-08-22 09:18:05,985::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000215808 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:05,985::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) Message size is 159 >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) (97) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) (97) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) (97) Building response >Thread-17::DEBUG::2014-08-22 09:18:05,986::__init__::225::IOProcess::(_processLogs) (97) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:05,987::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:18:05,988::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Dummy-45::DEBUG::2014-08-22 09:18:06,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:06,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121066 s, 84.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:08,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:08,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011753 s, 87.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:09,945::task::595::Storage.TaskManager.Task::(_updateState) Task=`28e9b7a0-6f5b-4cba-bf18-b4090d0e2ff0`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:09,945::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:18:09,945::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000215808', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000271423', 'lastCheck': '4.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:09,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`28e9b7a0-6f5b-4cba-bf18-b4090d0e2ff0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000215808', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000271423', 'lastCheck': '4.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:09,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`28e9b7a0-6f5b-4cba-bf18-b4090d0e2ff0`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:09,946::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:09,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:09,946::task::993::Storage.TaskManager.Task::(_decref) Task=`28e9b7a0-6f5b-4cba-bf18-b4090d0e2ff0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:10,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:10,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117844 s, 86.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:11,242::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:18:11,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`84dbbb07-4722-415d-9f47-c6d5306606a2`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:11,242::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:18:11,242::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:11,243::task::1191::Storage.TaskManager.Task::(prepare) Task=`84dbbb07-4722-415d-9f47-c6d5306606a2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:11,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`84dbbb07-4722-415d-9f47-c6d5306606a2`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:11,243::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:11,243::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:11,243::task::993::Storage.TaskManager.Task::(_decref) Task=`84dbbb07-4722-415d-9f47-c6d5306606a2`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:18:11,269::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:18:11,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6dbe574-48ad-4f64-a0ac-a74ab2f70c81`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:18:11,269::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:18:11,269::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12fed61e-21b6-4029-9541-10fdb88a77ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:18:11,269::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:18:11,270::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:18:11,270::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12fed61e-21b6-4029-9541-10fdb88a77ec`::Granted request >Thread-46::DEBUG::2014-08-22 09:18:11,270::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e6dbe574-48ad-4f64-a0ac-a74ab2f70c81`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:18:11,270::task::993::Storage.TaskManager.Task::(_decref) Task=`e6dbe574-48ad-4f64-a0ac-a74ab2f70c81`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:18:11,272::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:11,272::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6dbe574-48ad-4f64-a0ac-a74ab2f70c81`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:11,272::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6dbe574-48ad-4f64-a0ac-a74ab2f70c81`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:18:11,272::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:18:11,272::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:18:11,272::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:18:11,273::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:18:11,273::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:18:11,273::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:18:11,273::task::993::Storage.TaskManager.Task::(_decref) Task=`e6dbe574-48ad-4f64-a0ac-a74ab2f70c81`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:12,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:12,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117157 s, 87.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:13,455::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call vmCont with ('61901e42-5ccc-43da-aaa7-60ed6bfbd5f5',) {} flowID [41490768] >Thread-18::DEBUG::2014-08-22 09:18:13,495::BindingXMLRPC::1134::vds::(wrapper) return vmCont with {'status': {'message': 'Done', 'code': 0}, 'output': ['']} >Dummy-45::DEBUG::2014-08-22 09:18:14,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:14,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115744 s, 88.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) (98) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:15,939::__init__::225::IOProcess::(_processLogs) (98) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:15,940::__init__::225::IOProcess::(_processLogs) DEBUG|(98) Building response >Thread-17::DEBUG::2014-08-22 09:18:15,940::__init__::225::IOProcess::(_processLogs) (98) Queuing response >Thread-22::DEBUG::2014-08-22 09:18:15,941::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:15,941::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:15,947::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-22::DEBUG::2014-08-22 09:18:15,948::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000201137 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:15,948::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,948::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:18:15,948::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,948::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,948::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,949::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,949::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,949::__init__::225::IOProcess::(_processLogs) (99) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:15,949::__init__::225::IOProcess::(_processLogs) (99) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:15,949::__init__::225::IOProcess::(_processLogs) DEBUG|(99) Building response >Thread-17::DEBUG::2014-08-22 09:18:15,950::__init__::225::IOProcess::(_processLogs) (99) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:15,950::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:15,950::__init__::225::IOProcess::(_processLogs) Sending response sized 327 >Thread-17::DEBUG::2014-08-22 09:18:15,950::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,951::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:18:15,951::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,951::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,951::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,951::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,951::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) (100) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) (100) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) (100) Building response >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) (100) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:18:15,952::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) (101) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) (101) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) (101) Building response >Thread-17::DEBUG::2014-08-22 09:18:15,953::__init__::225::IOProcess::(_processLogs) (101) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:15,954::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:15,954::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:15,954::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,954::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:18:15,954::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,954::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) (102) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) (102) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) (102) Building response >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) (102) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:15,955::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:15,988::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,988::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:15,988::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,988::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,988::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,989::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,989::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,989::__init__::225::IOProcess::(_processLogs) (103) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:15,989::__init__::225::IOProcess::(_processLogs) (103) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:15,989::__init__::225::IOProcess::(_processLogs) DEBUG|(103) Building response >Thread-21::DEBUG::2014-08-22 09:18:15,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:15,990::__init__::225::IOProcess::(_processLogs) (103) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:15,996::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:15,996::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:18:15,997::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000232981 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:15,997::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) (104) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:15,998::__init__::225::IOProcess::(_processLogs) (104) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:15,999::__init__::225::IOProcess::(_processLogs) DEBUG|(104) Building response >Thread-17::DEBUG::2014-08-22 09:18:16,000::__init__::225::IOProcess::(_processLogs) (104) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:16,000::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:16,000::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-18::DEBUG::2014-08-22 09:18:16,079::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call vmGetStats with ('61901e42-5ccc-43da-aaa7-60ed6bfbd5f5',) {} >Thread-18::DEBUG::2014-08-22 09:18:16,079::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'displayInfo': [{'tlsPort': u'5901', 'ipAddress': '10.34.62.204', 'type': 'spice', 'port': u'5900'}], 'hash': '-5479945782673424400', 'acpiEnable': 'true', 'pid': '13420', 'displayIp': '10.34.62.204', 'displayPort': u'5900', 'displaySecurePort': u'5901', 'timeOffset': '0', 'kvmEnable': 'true', 'network': {}, 'vmId': '61901e42-5ccc-43da-aaa7-60ed6bfbd5f5', 'vmType': 'kvm', 'disks': {}, 'monitorResponse': '0', 'elapsedTime': '234540', 'displayType': 'qxl', 'clientIp': ''}]} >Dummy-45::DEBUG::2014-08-22 09:18:16,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:17,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119081 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:19,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:19,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116996 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:21,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:21,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116533 s, 87.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:21,321::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:18:21,321::task::595::Storage.TaskManager.Task::(_updateState) Task=`e61f358d-dbb0-47af-873f-c389f69f5ce9`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:21,321::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:18:21,321::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:21,322::task::1191::Storage.TaskManager.Task::(prepare) Task=`e61f358d-dbb0-47af-873f-c389f69f5ce9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:21,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`e61f358d-dbb0-47af-873f-c389f69f5ce9`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:21,322::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:21,322::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:21,322::task::993::Storage.TaskManager.Task::(_decref) Task=`e61f358d-dbb0-47af-873f-c389f69f5ce9`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:18:21,359::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:18:21,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a8c9a71-be40-4c7c-8d7f-a65481cc84a1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:18:21,359::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:18:21,359::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f95ac969-eb4f-4870-808a-938f20097a86`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:18:21,360::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:18:21,360::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:18:21,360::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f95ac969-eb4f-4870-808a-938f20097a86`::Granted request >Thread-46::DEBUG::2014-08-22 09:18:21,360::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1a8c9a71-be40-4c7c-8d7f-a65481cc84a1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:18:21,360::task::993::Storage.TaskManager.Task::(_decref) Task=`1a8c9a71-be40-4c7c-8d7f-a65481cc84a1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:18:21,362::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:21,362::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a8c9a71-be40-4c7c-8d7f-a65481cc84a1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:21,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a8c9a71-be40-4c7c-8d7f-a65481cc84a1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:18:21,362::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:18:21,362::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:18:21,363::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:18:21,363::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:18:21,363::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:18:21,363::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:18:21,363::task::993::Storage.TaskManager.Task::(_decref) Task=`1a8c9a71-be40-4c7c-8d7f-a65481cc84a1`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:18:21,853::BindingXMLRPC::1127::vds::(wrapper) client [10.34.63.60]::call vmSetTicket with ('61901e42-5ccc-43da-aaa7-60ed6bfbd5f5', '0V8fBdIvApfS', '120', 'disconnect', {'userName': 'admin', 'userId': 'fdfc627c-d875-11e0-90f0-83df133b58cc'}) {} flowID [13db2d] >Thread-18::DEBUG::2014-08-22 09:18:21,894::BindingXMLRPC::1134::vds::(wrapper) return vmSetTicket with {'status': {'message': 'Done', 'code': 0}} >Dummy-45::DEBUG::2014-08-22 09:18:23,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:23,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115981 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:25,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:25,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139043 s, 73.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:25,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`7cee90b8-042f-43f3-83fd-d6b44da6f4a7`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:25,240::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:18:25,240::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232981', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000201137', 'lastCheck': '9.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:25,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`7cee90b8-042f-43f3-83fd-d6b44da6f4a7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232981', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000201137', 'lastCheck': '9.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:25,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`7cee90b8-042f-43f3-83fd-d6b44da6f4a7`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:25,241::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:25,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:25,242::task::993::Storage.TaskManager.Task::(_decref) Task=`7cee90b8-042f-43f3-83fd-d6b44da6f4a7`::ref 0 aborting False >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,366::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 0 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,397::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 1 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,409::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 0 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,409::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 0 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,409::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 0 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,473::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 1 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,482::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 1 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,485::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 1 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,503::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 0 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,503::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 0 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,543::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 1 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:18:25,544::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 1 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >Thread-17::DEBUG::2014-08-22 09:18:25,955::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:25,955::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) (105) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:25,956::__init__::225::IOProcess::(_processLogs) (105) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:25,957::__init__::225::IOProcess::(_processLogs) DEBUG|(105) Building response >Thread-22::DEBUG::2014-08-22 09:18:25,957::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:25,957::__init__::225::IOProcess::(_processLogs) (105) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:25,963::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:25,963::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:18:25,964::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000241044 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:25,964::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:25,964::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:25,964::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:25,964::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:25,965::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:25,965::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:25,965::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:25,965::__init__::225::IOProcess::(_processLogs) (106) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:25,965::__init__::225::IOProcess::(_processLogs) (106) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:25,965::__init__::225::IOProcess::(_processLogs) DEBUG|(106) Building response >Thread-17::DEBUG::2014-08-22 09:18:25,966::__init__::225::IOProcess::(_processLogs) (106) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:25,966::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:25,966::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:25,967::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:25,967::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:18:25,967::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:25,967::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:25,967::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:25,967::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) (107) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) (107) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) (107) Building response >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) (107) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:18:25,968::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) (108) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) (108) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:25,969::__init__::225::IOProcess::(_processLogs) (108) Building response >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) (108) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:25,970::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:25,971::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:25,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:25,971::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:25,971::__init__::225::IOProcess::(_processLogs) (109) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:25,971::__init__::225::IOProcess::(_processLogs) (109) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:25,972::__init__::225::IOProcess::(_processLogs) (109) Building response >Thread-17::DEBUG::2014-08-22 09:18:25,972::__init__::225::IOProcess::(_processLogs) (109) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:25,972::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:25,972::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:26,000::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:26,001::__init__::225::IOProcess::(_processLogs) (110) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:26,001::__init__::225::IOProcess::(_processLogs) (110) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:26,001::__init__::225::IOProcess::(_processLogs) DEBUG|(110) Building response >Thread-21::DEBUG::2014-08-22 09:18:26,002::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:26,002::__init__::225::IOProcess::(_processLogs) (110) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:26,007::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:26,007::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:18:26,008::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000180688 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:26,008::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:26,008::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) (111) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:26,009::__init__::225::IOProcess::(_processLogs) (111) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:26,010::__init__::225::IOProcess::(_processLogs) DEBUG|(111) Building response >Thread-17::DEBUG::2014-08-22 09:18:26,011::__init__::225::IOProcess::(_processLogs) (111) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:26,011::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:26,011::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:18:27,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:27,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118122 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:29,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:29,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115923 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:31,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:31,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117964 s, 86.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:31,678::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:18:31,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f9d7851-4e8e-4f1b-8f25-1276895f5c67`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:31,679::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:18:31,679::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:31,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f9d7851-4e8e-4f1b-8f25-1276895f5c67`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:31,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f9d7851-4e8e-4f1b-8f25-1276895f5c67`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:31,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:31,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:31,679::task::993::Storage.TaskManager.Task::(_decref) Task=`2f9d7851-4e8e-4f1b-8f25-1276895f5c67`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:18:31,704::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:18:31,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`2df38a8d-1886-4771-95ac-86e5d7783ccb`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:18:31,704::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:18:31,704::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`efb66dbf-da07-40b6-929c-473dec265cdd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:18:31,704::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:18:31,705::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:18:31,705::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`efb66dbf-da07-40b6-929c-473dec265cdd`::Granted request >Thread-46::DEBUG::2014-08-22 09:18:31,705::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2df38a8d-1886-4771-95ac-86e5d7783ccb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:18:31,705::task::993::Storage.TaskManager.Task::(_decref) Task=`2df38a8d-1886-4771-95ac-86e5d7783ccb`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:18:31,707::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:31,707::task::1191::Storage.TaskManager.Task::(prepare) Task=`2df38a8d-1886-4771-95ac-86e5d7783ccb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:31,707::task::595::Storage.TaskManager.Task::(_updateState) Task=`2df38a8d-1886-4771-95ac-86e5d7783ccb`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:18:31,707::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:18:31,708::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:18:31,708::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:18:31,708::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:18:31,708::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:18:31,708::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:18:31,708::task::993::Storage.TaskManager.Task::(_decref) Task=`2df38a8d-1886-4771-95ac-86e5d7783ccb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:33,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:33,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118553 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:35,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:35,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118508 s, 86.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) (112) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:35,972::__init__::225::IOProcess::(_processLogs) (112) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:35,973::__init__::225::IOProcess::(_processLogs) DEBUG|(112) Building response >Thread-17::DEBUG::2014-08-22 09:18:35,974::__init__::225::IOProcess::(_processLogs) (112) Queuing response >Thread-22::DEBUG::2014-08-22 09:18:35,974::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:35,975::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:35,980::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:18:35,981::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000255349 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:35,982::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:35,983::__init__::225::IOProcess::(_processLogs) (113) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:35,983::__init__::225::IOProcess::(_processLogs) (113) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:35,983::__init__::225::IOProcess::(_processLogs) DEBUG|(113) Building response >Thread-17::DEBUG::2014-08-22 09:18:35,984::__init__::225::IOProcess::(_processLogs) (113) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:35,984::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:35,984::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:35,984::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:35,984::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:18:35,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:35,985::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:35,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:35,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:35,986::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:35,986::__init__::225::IOProcess::(_processLogs) (114) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:35,986::__init__::225::IOProcess::(_processLogs) (114) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:35,986::__init__::225::IOProcess::(_processLogs) (114) Building response >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) (114) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:35,987::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) (115) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) (115) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) (115) Building response >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) (115) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:35,988::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:35,989::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:35,989::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:35,989::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:18:35,989::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:35,989::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:35,990::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:35,990::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:35,990::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:35,990::__init__::225::IOProcess::(_processLogs) (116) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:35,990::__init__::225::IOProcess::(_processLogs) (116) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:35,990::__init__::225::IOProcess::(_processLogs) (116) Building response >Thread-17::DEBUG::2014-08-22 09:18:35,991::__init__::225::IOProcess::(_processLogs) (116) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:35,991::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:35,991::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:36,011::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:36,011::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:36,011::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:36,012::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:36,012::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:18:36,012::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:36,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:36,019::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:18:36,019::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000223386 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:36,020::__init__::225::IOProcess::(_processLogs) (117) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:36,020::__init__::225::IOProcess::(_processLogs) (117) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:36,020::__init__::225::IOProcess::(_processLogs) (117) Building response >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) (117) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:36,021::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:36,022::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:36,022::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:36,022::__init__::225::IOProcess::(_processLogs) (118) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:36,022::__init__::225::IOProcess::(_processLogs) (118) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:36,022::__init__::225::IOProcess::(_processLogs) DEBUG|(118) Building response >Thread-17::DEBUG::2014-08-22 09:18:36,023::__init__::225::IOProcess::(_processLogs) (118) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:36,023::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:36,023::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:18:37,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:37,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120672 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:39,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:39,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011642 s, 88.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:40,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`f76f97e7-14e5-4e05-a822-452beea70055`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:40,470::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:18:40,471::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223386', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000255349', 'lastCheck': '4.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:40,471::task::1191::Storage.TaskManager.Task::(prepare) Task=`f76f97e7-14e5-4e05-a822-452beea70055`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223386', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000255349', 'lastCheck': '4.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:40,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`f76f97e7-14e5-4e05-a822-452beea70055`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:40,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:40,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:40,471::task::993::Storage.TaskManager.Task::(_decref) Task=`f76f97e7-14e5-4e05-a822-452beea70055`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:41,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:41,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012874 s, 79.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:41,757::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:18:41,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`101f5eed-4a06-435d-8461-c430f6ff72ad`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:41,757::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:18:41,758::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:41,758::task::1191::Storage.TaskManager.Task::(prepare) Task=`101f5eed-4a06-435d-8461-c430f6ff72ad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:41,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`101f5eed-4a06-435d-8461-c430f6ff72ad`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:41,758::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:41,758::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:41,758::task::993::Storage.TaskManager.Task::(_decref) Task=`101f5eed-4a06-435d-8461-c430f6ff72ad`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:18:41,791::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:18:41,791::task::595::Storage.TaskManager.Task::(_updateState) Task=`2220a7e0-7751-4a23-adaf-f7eac5ee2e72`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:18:41,791::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:18:41,792::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`189935bc-40ae-4e2b-8a14-49c475441a31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:18:41,792::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:18:41,792::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:18:41,792::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`189935bc-40ae-4e2b-8a14-49c475441a31`::Granted request >Thread-46::DEBUG::2014-08-22 09:18:41,792::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2220a7e0-7751-4a23-adaf-f7eac5ee2e72`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:18:41,792::task::993::Storage.TaskManager.Task::(_decref) Task=`2220a7e0-7751-4a23-adaf-f7eac5ee2e72`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:18:41,794::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:41,795::task::1191::Storage.TaskManager.Task::(prepare) Task=`2220a7e0-7751-4a23-adaf-f7eac5ee2e72`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:41,795::task::595::Storage.TaskManager.Task::(_updateState) Task=`2220a7e0-7751-4a23-adaf-f7eac5ee2e72`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:18:41,795::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:18:41,795::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:18:41,795::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:18:41,795::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:18:41,795::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:18:41,795::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:18:41,795::task::993::Storage.TaskManager.Task::(_decref) Task=`2220a7e0-7751-4a23-adaf-f7eac5ee2e72`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:43,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:43,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116275 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:45,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:45,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115796 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) (119) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:45,991::__init__::225::IOProcess::(_processLogs) (119) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:45,992::__init__::225::IOProcess::(_processLogs) DEBUG|(119) Building response >Thread-22::DEBUG::2014-08-22 09:18:45,993::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:45,993::__init__::225::IOProcess::(_processLogs) (119) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:45,999::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:18:46,000::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0001856 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:46,000::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:46,001::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:46,002::__init__::225::IOProcess::(_processLogs) (120) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:46,002::__init__::225::IOProcess::(_processLogs) (120) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:46,002::__init__::225::IOProcess::(_processLogs) (120) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:46,003::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:46,003::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:46,003::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:18:46,003::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:46,004::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:46,004::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:46,004::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:46,004::__init__::225::IOProcess::(_processLogs) (121) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:46,004::__init__::225::IOProcess::(_processLogs) (121) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:46,004::__init__::225::IOProcess::(_processLogs) (121) Building response >Thread-17::DEBUG::2014-08-22 09:18:46,005::__init__::225::IOProcess::(_processLogs) (121) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:46,005::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:46,005::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:46,005::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:18:46,005::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:46,005::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) (122) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) (122) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) (122) Building response >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) (122) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:46,006::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:46,007::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:18:46,007::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:46,007::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:46,008::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:46,008::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:46,008::__init__::225::IOProcess::(_processLogs) (123) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:46,008::__init__::225::IOProcess::(_processLogs) (123) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:46,008::__init__::225::IOProcess::(_processLogs) (123) Building response >Thread-17::DEBUG::2014-08-22 09:18:46,008::__init__::225::IOProcess::(_processLogs) (123) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:46,009::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:46,009::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:46,024::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:46,024::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:46,024::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:46,024::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:46,024::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:18:46,025::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:46,025::__init__::225::IOProcess::(_processLogs) (124) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:46,031::__init__::225::IOProcess::(_processLogs) (124) Got request for method 'statvfs' >Thread-21::DEBUG::2014-08-22 09:18:46,032::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00019579 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:46,032::__init__::225::IOProcess::(_processLogs) (124) Building response >Thread-17::DEBUG::2014-08-22 09:18:46,033::__init__::225::IOProcess::(_processLogs) (124) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:46,033::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:46,033::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:46,033::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:46,033::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:46,033::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:46,034::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:46,034::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:46,034::__init__::225::IOProcess::(_processLogs) (125) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:46,034::__init__::225::IOProcess::(_processLogs) (125) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:46,034::__init__::225::IOProcess::(_processLogs) (125) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:46,035::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:46,035::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:18:47,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:47,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120014 s, 85.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:18:49,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:49,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116853 s, 87.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:51,867::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:18:51,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`20a57e80-9737-4538-9f0b-49301f9bed64`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:51,867::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:18:51,868::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:51,868::task::1191::Storage.TaskManager.Task::(prepare) Task=`20a57e80-9737-4538-9f0b-49301f9bed64`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:18:51,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`20a57e80-9737-4538-9f0b-49301f9bed64`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:51,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:51,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:51,868::task::993::Storage.TaskManager.Task::(_decref) Task=`20a57e80-9737-4538-9f0b-49301f9bed64`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:51,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:51,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170407 s, 60.1 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:18:51,958::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:18:51,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`9442b2e4-2194-4c32-9a11-f4cffc8b1329`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:18:51,958::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:18:51,959::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`17aaa704-35c5-403c-9979-53226aa4a03c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:18:51,959::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:18:51,959::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:18:51,959::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`17aaa704-35c5-403c-9979-53226aa4a03c`::Granted request >Thread-46::DEBUG::2014-08-22 09:18:51,959::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9442b2e4-2194-4c32-9a11-f4cffc8b1329`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:18:51,959::task::993::Storage.TaskManager.Task::(_decref) Task=`9442b2e4-2194-4c32-9a11-f4cffc8b1329`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:18:51,963::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:51,963::task::1191::Storage.TaskManager.Task::(prepare) Task=`9442b2e4-2194-4c32-9a11-f4cffc8b1329`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:18:51,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`9442b2e4-2194-4c32-9a11-f4cffc8b1329`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:18:51,963::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:18:51,963::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:18:51,963::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:18:51,963::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:18:51,963::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:18:51,964::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:18:51,964::task::993::Storage.TaskManager.Task::(_decref) Task=`9442b2e4-2194-4c32-9a11-f4cffc8b1329`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:53,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:53,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011676 s, 87.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:18:55,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f6d8395-afa3-4ebe-8eee-fbfd94687c48`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:18:55,719::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:18:55,720::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00019579', 'lastCheck': '9.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0001856', 'lastCheck': '9.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:55,720::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f6d8395-afa3-4ebe-8eee-fbfd94687c48`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00019579', 'lastCheck': '9.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0001856', 'lastCheck': '9.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:18:55,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f6d8395-afa3-4ebe-8eee-fbfd94687c48`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:18:55,720::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:18:55,720::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:18:55,720::task::993::Storage.TaskManager.Task::(_decref) Task=`4f6d8395-afa3-4ebe-8eee-fbfd94687c48`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:18:55,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:56,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115917 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:56,029::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:56,030::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,030::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:18:56,031::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:56,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:56,034::__init__::225::IOProcess::(_processLogs) (126) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:56,036::__init__::225::IOProcess::(_processLogs) (126) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 09:18:56,036::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:56,037::__init__::225::IOProcess::(_processLogs) (126) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,043::__init__::225::IOProcess::(_processLogs) (126) Queuing response >Thread-22::DEBUG::2014-08-22 09:18:56,044::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000194921 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:56,044::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,046::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:56,047::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:56,047::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,049::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:56,050::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:18:56,050::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:18:56,051::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:56,057::__init__::225::IOProcess::(_processLogs) (127) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:18:56,058::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199161 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:18:56,058::__init__::225::IOProcess::(_processLogs) (127) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:56,060::__init__::225::IOProcess::(_processLogs) (127) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,060::__init__::225::IOProcess::(_processLogs) (127) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:56,061::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,061::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:56,062::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:18:56,063::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,064::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:56,066::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:56,066::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:56,066::__init__::225::IOProcess::(_processLogs) (128) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:56,067::__init__::225::IOProcess::(_processLogs) (128) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:56,067::__init__::225::IOProcess::(_processLogs) (128) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,068::__init__::225::IOProcess::(_processLogs) (128) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:56,068::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,068::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:56,069::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:18:56,069::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,071::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:56,072::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:56,073::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:56,073::__init__::225::IOProcess::(_processLogs) (129) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:18:56,074::__init__::225::IOProcess::(_processLogs) (129) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:18:56,074::__init__::225::IOProcess::(_processLogs) (129) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,074::__init__::225::IOProcess::(_processLogs) (129) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:56,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,075::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:18:56,076::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:18:56,076::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,078::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:56,078::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:56,079::__init__::225::IOProcess::(_processLogs) (130) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:56,079::__init__::225::IOProcess::(_processLogs) (130) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:56,079::__init__::225::IOProcess::(_processLogs) (130) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,080::__init__::225::IOProcess::(_processLogs) (130) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:56,080::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,081::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:56,081::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:18:56,082::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,083::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:56,084::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:56,084::__init__::225::IOProcess::(_processLogs) (131) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:56,084::__init__::225::IOProcess::(_processLogs) (131) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:56,085::__init__::225::IOProcess::(_processLogs) (131) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,085::__init__::225::IOProcess::(_processLogs) (131) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:56,085::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,086::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:18:56,087::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:18:56,087::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:18:56,088::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:18:56,088::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:18:56,089::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:18:56,089::__init__::225::IOProcess::(_processLogs) (132) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:18:56,090::__init__::225::IOProcess::(_processLogs) (132) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:18:56,090::__init__::225::IOProcess::(_processLogs) (132) Building response >Thread-17::DEBUG::2014-08-22 09:18:56,090::__init__::225::IOProcess::(_processLogs) (132) Queuing response >Thread-17::DEBUG::2014-08-22 09:18:56,091::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:18:56,091::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:18:58,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:18:58,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120216 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:00,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:00,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120626 s, 84.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:19:02,016::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:19:02,017::task::595::Storage.TaskManager.Task::(_updateState) Task=`378875e9-e6e4-4f9c-894c-860cbeb18d67`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:19:02,018::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:19:02,019::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:19:02,020::task::1191::Storage.TaskManager.Task::(prepare) Task=`378875e9-e6e4-4f9c-894c-860cbeb18d67`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:19:02,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`378875e9-e6e4-4f9c-894c-860cbeb18d67`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:19:02,021::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:19:02,022::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:19:02,022::task::993::Storage.TaskManager.Task::(_decref) Task=`378875e9-e6e4-4f9c-894c-860cbeb18d67`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:19:02,050::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:19:02,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b184a6b-6fcd-49ef-81e2-2c06999e8b69`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:19:02,051::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:19:02,053::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3b6cace9-a392-485a-b5ca-5addffdaee02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:19:02,054::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:19:02,055::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:19:02,055::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3b6cace9-a392-485a-b5ca-5addffdaee02`::Granted request >Thread-46::DEBUG::2014-08-22 09:19:02,057::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0b184a6b-6fcd-49ef-81e2-2c06999e8b69`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:19:02,057::task::993::Storage.TaskManager.Task::(_decref) Task=`0b184a6b-6fcd-49ef-81e2-2c06999e8b69`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:19:02,069::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:02,069::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b184a6b-6fcd-49ef-81e2-2c06999e8b69`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:02,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b184a6b-6fcd-49ef-81e2-2c06999e8b69`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:19:02,070::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:19:02,071::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:19:02,071::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:19:02,072::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:19:02,072::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:19:02,073::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:19:02,073::task::993::Storage.TaskManager.Task::(_decref) Task=`0b184a6b-6fcd-49ef-81e2-2c06999e8b69`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:02,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:02,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190081 s, 53.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:04,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:04,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116018 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:06,073::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:06,073::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:06,073::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,073::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,073::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,073::__init__::225::IOProcess::(_processLogs) (133) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:06,074::__init__::225::IOProcess::(_processLogs) (133) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:06,074::__init__::225::IOProcess::(_processLogs) (133) Queuing response >Thread-21::DEBUG::2014-08-22 09:19:06,075::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:06,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,081::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:19:06,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000180944 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:06,082::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:06,082::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:06,082::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,083::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,083::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,083::__init__::225::IOProcess::(_processLogs) (134) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:06,083::__init__::225::IOProcess::(_processLogs) (134) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:06,083::__init__::225::IOProcess::(_processLogs) (134) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:06,084::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,084::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:06,094::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:06,094::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:06,094::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,094::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,094::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,095::__init__::225::IOProcess::(_processLogs) (135) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:06,095::__init__::225::IOProcess::(_processLogs) (135) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:06,095::__init__::225::IOProcess::(_processLogs) (135) Queuing response >Thread-22::DEBUG::2014-08-22 09:19:06,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:06,096::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,101::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:19:06,102::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000242276 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) (136) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:06,103::__init__::225::IOProcess::(_processLogs) (136) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:06,104::__init__::225::IOProcess::(_processLogs) (136) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:06,105::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,105::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:06,105::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:19:06,105::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) (137) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) (137) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) (137) Building response >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) (137) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,106::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:06,107::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:19:06,107::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:06,107::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,107::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,107::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) (138) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) (138) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) (138) Building response >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) (138) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:19:06,108::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:06,109::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:06,109::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:06,109::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:06,109::__init__::225::IOProcess::(_processLogs) (139) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:06,109::__init__::225::IOProcess::(_processLogs) (139) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:06,109::__init__::225::IOProcess::(_processLogs) (139) Building response >Thread-17::DEBUG::2014-08-22 09:19:06,110::__init__::225::IOProcess::(_processLogs) (139) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:06,110::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:06,110::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:19:06,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:06,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011817 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:08,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:08,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119267 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:10,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:10,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117253 s, 87.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:19:10,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`b93da837-f52a-4b32-acb5-1b51e5309c06`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:19:10,904::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:19:10,905::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180944', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000242276', 'lastCheck': '4.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:19:10,905::task::1191::Storage.TaskManager.Task::(prepare) Task=`b93da837-f52a-4b32-acb5-1b51e5309c06`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180944', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000242276', 'lastCheck': '4.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:19:10,905::task::595::Storage.TaskManager.Task::(_updateState) Task=`b93da837-f52a-4b32-acb5-1b51e5309c06`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:19:10,905::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:19:10,905::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:19:10,905::task::993::Storage.TaskManager.Task::(_decref) Task=`b93da837-f52a-4b32-acb5-1b51e5309c06`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 09:19:12,134::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:19:12,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`c863af23-a993-4f9e-adf3-92e2a7226a3f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:19:12,134::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:19:12,134::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:19:12,134::task::1191::Storage.TaskManager.Task::(prepare) Task=`c863af23-a993-4f9e-adf3-92e2a7226a3f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:19:12,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`c863af23-a993-4f9e-adf3-92e2a7226a3f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:19:12,134::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:19:12,135::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:19:12,135::task::993::Storage.TaskManager.Task::(_decref) Task=`c863af23-a993-4f9e-adf3-92e2a7226a3f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:19:12,169::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:19:12,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`f374a4c2-acc1-4539-ab28-a2a6e4641fa7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:19:12,169::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:19:12,170::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`44f866ac-ae4f-4dd1-a6b3-6959fe8cc18b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:19:12,170::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:19:12,170::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:19:12,170::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`44f866ac-ae4f-4dd1-a6b3-6959fe8cc18b`::Granted request >Thread-46::DEBUG::2014-08-22 09:19:12,170::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f374a4c2-acc1-4539-ab28-a2a6e4641fa7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:19:12,170::task::993::Storage.TaskManager.Task::(_decref) Task=`f374a4c2-acc1-4539-ab28-a2a6e4641fa7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:19:12,172::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:12,172::task::1191::Storage.TaskManager.Task::(prepare) Task=`f374a4c2-acc1-4539-ab28-a2a6e4641fa7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:12,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`f374a4c2-acc1-4539-ab28-a2a6e4641fa7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:19:12,172::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:19:12,172::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:19:12,173::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:19:12,173::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:19:12,173::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:19:12,173::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:19:12,173::task::993::Storage.TaskManager.Task::(_decref) Task=`f374a4c2-acc1-4539-ab28-a2a6e4641fa7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:12,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:12,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115528 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:14,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:14,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117649 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:16,085::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:16,085::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:16,085::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:16,085::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,086::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,086::__init__::225::IOProcess::(_processLogs) (140) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:16,086::__init__::225::IOProcess::(_processLogs) (140) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:16,086::__init__::225::IOProcess::(_processLogs) (140) Queuing response >Thread-21::DEBUG::2014-08-22 09:19:16,087::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:16,087::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,093::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:19:16,094::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000177217 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:16,094::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:16,095::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:16,095::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,095::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,095::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,095::__init__::225::IOProcess::(_processLogs) (141) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:16,095::__init__::225::IOProcess::(_processLogs) (141) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:16,096::__init__::225::IOProcess::(_processLogs) (141) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:16,096::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,096::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:16,109::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:16,110::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:16,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,110::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,110::__init__::225::IOProcess::(_processLogs) (142) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:16,110::__init__::225::IOProcess::(_processLogs) (142) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:16,111::__init__::225::IOProcess::(_processLogs) (142) Queuing response >Thread-22::DEBUG::2014-08-22 09:19:16,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:16,112::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,117::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:19:16,118::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000169324 s, 3.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) (143) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:16,119::__init__::225::IOProcess::(_processLogs) (143) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:16,120::__init__::225::IOProcess::(_processLogs) (143) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) (144) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:16,121::__init__::225::IOProcess::(_processLogs) (144) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:16,122::__init__::225::IOProcess::(_processLogs) (144) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:16,122::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,122::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:16,122::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:19:16,122::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) (145) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) (145) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) (145) Building response >Thread-17::DEBUG::2014-08-22 09:19:16,123::__init__::225::IOProcess::(_processLogs) (145) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:16,124::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,124::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:16,124::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:19:16,124::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:16,124::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:16,124::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:16,125::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:16,125::__init__::225::IOProcess::(_processLogs) (146) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:16,125::__init__::225::IOProcess::(_processLogs) (146) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:16,125::__init__::225::IOProcess::(_processLogs) (146) Building response >Thread-17::DEBUG::2014-08-22 09:19:16,125::__init__::225::IOProcess::(_processLogs) (146) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:16,125::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:16,126::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:19:16,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:16,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120502 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:18,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:18,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01182 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:20,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:20,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113886 s, 89.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 09:19:22,213::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 09:19:22,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`73497b64-d326-417b-a777-a2d80570b823`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:19:22,214::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 09:19:22,214::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:19:22,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`73497b64-d326-417b-a777-a2d80570b823`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 09:19:22,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`73497b64-d326-417b-a777-a2d80570b823`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:19:22,214::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:19:22,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:19:22,215::task::993::Storage.TaskManager.Task::(_decref) Task=`73497b64-d326-417b-a777-a2d80570b823`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:19:22,264::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:19:22,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8918e37-28af-4ec6-b182-511bc5321fe9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:19:22,264::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:19:22,265::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`27d1f871-ebae-4420-8a91-e48b32e91e80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:19:22,265::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:19:22,265::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:19:22,265::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`27d1f871-ebae-4420-8a91-e48b32e91e80`::Granted request >Thread-46::DEBUG::2014-08-22 09:19:22,265::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e8918e37-28af-4ec6-b182-511bc5321fe9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:19:22,265::task::993::Storage.TaskManager.Task::(_decref) Task=`e8918e37-28af-4ec6-b182-511bc5321fe9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:19:22,267::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:22,267::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8918e37-28af-4ec6-b182-511bc5321fe9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:22,267::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8918e37-28af-4ec6-b182-511bc5321fe9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:19:22,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:19:22,268::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:19:22,268::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:19:22,268::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:19:22,268::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:19:22,268::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:19:22,268::task::993::Storage.TaskManager.Task::(_decref) Task=`e8918e37-28af-4ec6-b182-511bc5321fe9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:22,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:22,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117545 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:24,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:24,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116051 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:26,097::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:26,097::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:26,097::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,097::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,097::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,097::__init__::225::IOProcess::(_processLogs) (147) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:26,098::__init__::225::IOProcess::(_processLogs) (147) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:26,098::__init__::225::IOProcess::(_processLogs) (147) Queuing response >Thread-21::DEBUG::2014-08-22 09:19:26,099::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:26,099::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,105::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:19:26,106::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219409 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:26,106::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:26,107::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:26,107::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,107::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,107::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,107::__init__::225::IOProcess::(_processLogs) (148) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:26,107::__init__::225::IOProcess::(_processLogs) (148) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:26,108::__init__::225::IOProcess::(_processLogs) (148) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:26,109::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,109::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-18::DEBUG::2014-08-22 09:19:26,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`42a91dd9-d468-4283-aec5-f376c4f6405d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 09:19:26,116::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 09:19:26,116::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219409', 'lastCheck': '0.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000169324', 'lastCheck': '10.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:19:26,116::task::1191::Storage.TaskManager.Task::(prepare) Task=`42a91dd9-d468-4283-aec5-f376c4f6405d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219409', 'lastCheck': '0.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000169324', 'lastCheck': '10.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 09:19:26,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`42a91dd9-d468-4283-aec5-f376c4f6405d`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 09:19:26,117::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 09:19:26,117::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 09:19:26,117::task::993::Storage.TaskManager.Task::(_decref) Task=`42a91dd9-d468-4283-aec5-f376c4f6405d`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:19:26,125::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:26,125::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:26,126::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,126::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,126::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,126::__init__::225::IOProcess::(_processLogs) (149) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:26,126::__init__::225::IOProcess::(_processLogs) (149) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:26,126::__init__::225::IOProcess::(_processLogs) (149) Queuing response >Thread-22::DEBUG::2014-08-22 09:19:26,127::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:26,127::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,133::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:19:26,134::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000155757 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) (150) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:26,135::__init__::225::IOProcess::(_processLogs) (150) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:26,136::__init__::225::IOProcess::(_processLogs) (150) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:26,137::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,137::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:26,137::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:19:26,137::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:26,137::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) (151) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) (151) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) (151) Building response >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) (151) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,138::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:26,139::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:19:26,139::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:26,139::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,139::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,139::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,139::__init__::225::IOProcess::(_processLogs) (152) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) (152) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) (152) Building response >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) (152) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:19:26,140::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:26,141::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:26,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:26,144::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:26,145::__init__::225::IOProcess::(_processLogs) (153) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:26,146::__init__::225::IOProcess::(_processLogs) (153) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:26,147::__init__::225::IOProcess::(_processLogs) (153) Building response >Thread-17::DEBUG::2014-08-22 09:19:26,147::__init__::225::IOProcess::(_processLogs) (153) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:26,147::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:26,147::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:19:26,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:26,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012109 s, 84.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:28,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:28,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011681 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:30,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:31,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116276 s, 88.1 MB/s\n'; <rc> = 0 >Detector thread::DEBUG::2014-08-22 09:19:32,324::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.60:36904 >Detector thread::DEBUG::2014-08-22 09:19:32,324::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.60:36904 >Detector thread::DEBUG::2014-08-22 09:19:32,324::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.34.63.60:36904 >Detector thread::DEBUG::2014-08-22 09:19:32,325::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.34.63.60', 36904) >Thread-47::DEBUG::2014-08-22 09:19:32,326::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:19:32,327::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f3ae8ce-170c-431c-a45c-1fe20454802d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:19:32,328::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:19:32,329::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:19:32,329::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f3ae8ce-170c-431c-a45c-1fe20454802d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:19:32,330::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f3ae8ce-170c-431c-a45c-1fe20454802d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:19:32,330::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:19:32,330::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:19:32,331::task::993::Storage.TaskManager.Task::(_decref) Task=`8f3ae8ce-170c-431c-a45c-1fe20454802d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:19:32,403::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:19:32,404::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a6df5c4-8ad0-4576-8fed-db9db7daf140`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:19:32,405::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:19:32,407::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a473d220-e928-4441-a1d0-27daa126fbbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:19:32,407::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:19:32,408::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:19:32,409::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a473d220-e928-4441-a1d0-27daa126fbbd`::Granted request >Thread-46::DEBUG::2014-08-22 09:19:32,411::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1a6df5c4-8ad0-4576-8fed-db9db7daf140`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:19:32,411::task::993::Storage.TaskManager.Task::(_decref) Task=`1a6df5c4-8ad0-4576-8fed-db9db7daf140`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:19:32,425::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:32,426::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a6df5c4-8ad0-4576-8fed-db9db7daf140`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:32,427::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a6df5c4-8ad0-4576-8fed-db9db7daf140`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:19:32,427::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:19:32,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:19:32,429::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:19:32,430::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:19:32,431::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:19:32,431::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:19:32,432::task::993::Storage.TaskManager.Task::(_decref) Task=`1a6df5c4-8ad0-4576-8fed-db9db7daf140`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:33,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:33,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143389 s, 71.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:35,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:35,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116637 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:36,109::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:36,109::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:36,109::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:36,109::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,110::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,110::__init__::225::IOProcess::(_processLogs) (154) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:36,110::__init__::225::IOProcess::(_processLogs) (154) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:36,110::__init__::225::IOProcess::(_processLogs) (154) Queuing response >Thread-21::DEBUG::2014-08-22 09:19:36,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:36,111::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:36,117::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:19:36,118::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000180395 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) (155) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:36,119::__init__::225::IOProcess::(_processLogs) (155) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:36,120::__init__::225::IOProcess::(_processLogs) (155) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:36,121::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:36,121::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) (156) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:36,143::__init__::225::IOProcess::(_processLogs) (156) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:36,144::__init__::225::IOProcess::(_processLogs) (156) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:36,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:19:36,144::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:36,145::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:19:36,150::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000154013 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) (157) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:36,152::__init__::225::IOProcess::(_processLogs) (157) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:36,153::__init__::225::IOProcess::(_processLogs) (157) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:36,153::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:36,154::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:36,154::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:19:36,154::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:36,154::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) (158) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) (158) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) (158) Building response >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) (158) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:19:36,155::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) (159) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) (159) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) (159) Building response >Thread-17::DEBUG::2014-08-22 09:19:36,156::__init__::225::IOProcess::(_processLogs) (159) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:36,157::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:36,157::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:36,157::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:19:36,157::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:36,157::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) (160) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) (160) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) (160) Building response >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) (160) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:36,158::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:36,159::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:19:37,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:37,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120441 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:39,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:39,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114926 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:41,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:41,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011851 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:19:41,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6c7d4e6-2c61-4d52-bce8-50a2f303daa2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:19:41,359::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:19:41,360::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180395', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000154013', 'lastCheck': '5.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:19:41,360::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6c7d4e6-2c61-4d52-bce8-50a2f303daa2`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180395', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000154013', 'lastCheck': '5.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:19:41,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6c7d4e6-2c61-4d52-bce8-50a2f303daa2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:19:41,360::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:19:41,360::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:19:41,360::task::993::Storage.TaskManager.Task::(_decref) Task=`a6c7d4e6-2c61-4d52-bce8-50a2f303daa2`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:19:42,512::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:19:42,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`f840e8b2-3124-4e72-a461-e70388376122`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:19:42,512::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:19:42,512::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:19:42,512::task::1191::Storage.TaskManager.Task::(prepare) Task=`f840e8b2-3124-4e72-a461-e70388376122`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:19:42,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`f840e8b2-3124-4e72-a461-e70388376122`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:19:42,512::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:19:42,513::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:19:42,513::task::993::Storage.TaskManager.Task::(_decref) Task=`f840e8b2-3124-4e72-a461-e70388376122`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:19:42,557::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:19:42,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`251b69a5-d45c-4d89-8dcd-ace4f725152c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:19:42,557::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:19:42,557::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8a5d925a-1951-4b85-b36f-1c05f41f948a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:19:42,557::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:19:42,557::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:19:42,557::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8a5d925a-1951-4b85-b36f-1c05f41f948a`::Granted request >Thread-46::DEBUG::2014-08-22 09:19:42,558::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`251b69a5-d45c-4d89-8dcd-ace4f725152c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:19:42,558::task::993::Storage.TaskManager.Task::(_decref) Task=`251b69a5-d45c-4d89-8dcd-ace4f725152c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:19:42,560::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:42,560::task::1191::Storage.TaskManager.Task::(prepare) Task=`251b69a5-d45c-4d89-8dcd-ace4f725152c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:42,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`251b69a5-d45c-4d89-8dcd-ace4f725152c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:19:42,560::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:19:42,560::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:19:42,560::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:19:42,560::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:19:42,560::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:19:42,560::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:19:42,560::task::993::Storage.TaskManager.Task::(_decref) Task=`251b69a5-d45c-4d89-8dcd-ace4f725152c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:43,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:43,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116427 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:45,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:45,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011708 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:46,121::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:46,121::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:46,121::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,122::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,122::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,122::__init__::225::IOProcess::(_processLogs) (161) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:46,122::__init__::225::IOProcess::(_processLogs) (161) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:46,122::__init__::225::IOProcess::(_processLogs) (161) Queuing response >Thread-21::DEBUG::2014-08-22 09:19:46,123::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:46,123::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:46,129::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:19:46,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000218501 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) (162) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:46,131::__init__::225::IOProcess::(_processLogs) (162) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:46,132::__init__::225::IOProcess::(_processLogs) (162) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:46,133::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:46,133::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:46,158::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:46,158::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:46,159::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,159::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,159::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,159::__init__::225::IOProcess::(_processLogs) (163) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:46,159::__init__::225::IOProcess::(_processLogs) (163) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:46,159::__init__::225::IOProcess::(_processLogs) (163) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:46,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:19:46,160::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:46,161::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:19:46,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000303045 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:46,173::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:46,173::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:46,174::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,174::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,174::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,174::__init__::225::IOProcess::(_processLogs) (164) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:46,174::__init__::225::IOProcess::(_processLogs) (164) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:46,175::__init__::225::IOProcess::(_processLogs) (164) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:46,175::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:46,175::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:46,175::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:19:46,175::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:46,176::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,176::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,176::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,176::__init__::225::IOProcess::(_processLogs) (165) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:46,176::__init__::225::IOProcess::(_processLogs) (165) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:46,177::__init__::225::IOProcess::(_processLogs) (165) Building response >Thread-17::DEBUG::2014-08-22 09:19:46,177::__init__::225::IOProcess::(_processLogs) (165) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:46,177::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:46,177::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:46,177::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:19:46,177::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) (166) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) (166) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) (166) Building response >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) (166) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:46,178::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:46,179::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:19:46,179::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:46,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) (167) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) (167) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) (167) Building response >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) (167) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:46,180::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:19:47,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:47,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116237 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:49,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:49,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115575 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:51,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:51,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119302 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:19:52,626::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:19:52,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c2ee6d6-b84f-42c5-b836-a43b7c954662`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:19:52,627::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:19:52,627::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:19:52,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c2ee6d6-b84f-42c5-b836-a43b7c954662`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:19:52,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c2ee6d6-b84f-42c5-b836-a43b7c954662`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:19:52,627::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:19:52,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:19:52,628::task::993::Storage.TaskManager.Task::(_decref) Task=`9c2ee6d6-b84f-42c5-b836-a43b7c954662`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:19:52,673::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:19:52,673::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6dc1f5a-58f7-4b75-b948-38352f99282c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:19:52,673::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:19:52,674::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`df22f7a8-ddad-44c0-9241-a09616f519c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:19:52,674::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:19:52,674::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:19:52,675::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`df22f7a8-ddad-44c0-9241-a09616f519c7`::Granted request >Thread-46::DEBUG::2014-08-22 09:19:52,675::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f6dc1f5a-58f7-4b75-b948-38352f99282c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:19:52,675::task::993::Storage.TaskManager.Task::(_decref) Task=`f6dc1f5a-58f7-4b75-b948-38352f99282c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:19:52,677::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:52,677::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6dc1f5a-58f7-4b75-b948-38352f99282c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:19:52,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6dc1f5a-58f7-4b75-b948-38352f99282c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:19:52,677::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:19:52,677::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:19:52,677::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:19:52,677::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:19:52,677::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:19:52,678::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:19:52,678::task::993::Storage.TaskManager.Task::(_decref) Task=`f6dc1f5a-58f7-4b75-b948-38352f99282c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:53,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:53,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117417 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:55,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:55,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114992 s, 89.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:56,133::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:56,134::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:56,134::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,134::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,134::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,134::__init__::225::IOProcess::(_processLogs) (168) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:56,134::__init__::225::IOProcess::(_processLogs) (168) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:56,135::__init__::225::IOProcess::(_processLogs) (168) Queuing response >Thread-21::DEBUG::2014-08-22 09:19:56,135::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:56,135::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:56,141::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:19:56,142::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000222527 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:56,143::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:19:56,143::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:56,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,143::__init__::225::IOProcess::(_processLogs) (169) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:56,144::__init__::225::IOProcess::(_processLogs) (169) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:56,144::__init__::225::IOProcess::(_processLogs) (169) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:56,145::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:56,145::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:56,180::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:56,180::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:56,180::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,180::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,180::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,181::__init__::225::IOProcess::(_processLogs) (170) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:56,181::__init__::225::IOProcess::(_processLogs) (170) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:56,181::__init__::225::IOProcess::(_processLogs) (170) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:56,182::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:19:56,182::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:19:56,182::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:19:56,188::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199034 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:19:56,188::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:19:56,189::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:19:56,189::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,189::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,189::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,189::__init__::225::IOProcess::(_processLogs) (171) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:19:56,189::__init__::225::IOProcess::(_processLogs) (171) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:19:56,190::__init__::225::IOProcess::(_processLogs) (171) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:56,190::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:56,190::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:19:56,191::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:19:56,191::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:56,191::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,191::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,191::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) (172) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) (172) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) (172) Building response >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) (172) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,192::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,193::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,193::__init__::225::IOProcess::(_processLogs) (173) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:56,193::__init__::225::IOProcess::(_processLogs) (173) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:56,193::__init__::225::IOProcess::(_processLogs) (173) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:56,193::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:56,193::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) (174) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) (174) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:19:56,194::__init__::225::IOProcess::(_processLogs) (174) Queuing response >Thread-17::DEBUG::2014-08-22 09:19:56,195::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:19:56,195::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:19:56,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`6519b649-21cc-43ec-ac63-bbb031d8756e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:19:56,651::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:19:56,651::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000222527', 'lastCheck': '0.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199034', 'lastCheck': '0.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:19:56,651::task::1191::Storage.TaskManager.Task::(prepare) Task=`6519b649-21cc-43ec-ac63-bbb031d8756e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000222527', 'lastCheck': '0.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199034', 'lastCheck': '0.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:19:56,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`6519b649-21cc-43ec-ac63-bbb031d8756e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:19:56,651::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:19:56,651::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:19:56,651::task::993::Storage.TaskManager.Task::(_decref) Task=`6519b649-21cc-43ec-ac63-bbb031d8756e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:19:57,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:57,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011927 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:19:59,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:19:59,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114266 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:01,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:01,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114836 s, 89.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:02,747::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:20:02,748::task::595::Storage.TaskManager.Task::(_updateState) Task=`6706f5a3-f350-4529-90c1-bd3804395d7c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:02,749::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:20:02,750::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:02,750::task::1191::Storage.TaskManager.Task::(prepare) Task=`6706f5a3-f350-4529-90c1-bd3804395d7c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:02,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`6706f5a3-f350-4529-90c1-bd3804395d7c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:02,751::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:02,751::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:02,752::task::993::Storage.TaskManager.Task::(_decref) Task=`6706f5a3-f350-4529-90c1-bd3804395d7c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:20:02,833::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:20:02,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`986579ed-6a99-4357-9f28-48bffcd2ae0d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:20:02,837::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:20:02,839::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`45f01112-437a-48b3-9be5-078769861348`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:20:02,840::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:20:02,841::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:20:02,842::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`45f01112-437a-48b3-9be5-078769861348`::Granted request >Thread-46::DEBUG::2014-08-22 09:20:02,843::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`986579ed-6a99-4357-9f28-48bffcd2ae0d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:20:02,844::task::993::Storage.TaskManager.Task::(_decref) Task=`986579ed-6a99-4357-9f28-48bffcd2ae0d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:20:02,858::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:02,858::task::1191::Storage.TaskManager.Task::(prepare) Task=`986579ed-6a99-4357-9f28-48bffcd2ae0d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:02,859::task::595::Storage.TaskManager.Task::(_updateState) Task=`986579ed-6a99-4357-9f28-48bffcd2ae0d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:20:02,859::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:20:02,860::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:20:02,860::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:20:02,861::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:20:02,861::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:20:02,862::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:20:02,863::task::993::Storage.TaskManager.Task::(_decref) Task=`986579ed-6a99-4357-9f28-48bffcd2ae0d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:03,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:03,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115857 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:05,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:05,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141373 s, 72.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) (175) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:06,146::__init__::225::IOProcess::(_processLogs) (175) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:06,147::__init__::225::IOProcess::(_processLogs) (175) Queuing response >Thread-21::DEBUG::2014-08-22 09:20:06,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:06,147::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,153::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:20:06,154::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219096 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) (176) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:06,155::__init__::225::IOProcess::(_processLogs) (176) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:06,156::__init__::225::IOProcess::(_processLogs) (176) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:06,157::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,157::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) (177) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:06,196::__init__::225::IOProcess::(_processLogs) (177) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:06,197::__init__::225::IOProcess::(_processLogs) (177) Queuing response >Thread-22::DEBUG::2014-08-22 09:20:06,197::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:06,197::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,203::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:20:06,204::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000211333 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:06,204::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:06,204::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:06,204::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,205::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,205::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,205::__init__::225::IOProcess::(_processLogs) (178) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:06,205::__init__::225::IOProcess::(_processLogs) (178) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:06,205::__init__::225::IOProcess::(_processLogs) (178) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:06,206::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,206::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:06,206::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:20:06,206::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) (179) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) (179) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) (179) Building response >Thread-17::DEBUG::2014-08-22 09:20:06,207::__init__::225::IOProcess::(_processLogs) (179) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:06,208::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,208::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:06,208::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:20:06,208::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:06,208::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) (180) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) (180) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) (180) Building response >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) (180) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:20:06,209::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:06,210::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:06,210::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:06,210::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:06,210::__init__::225::IOProcess::(_processLogs) (181) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:06,210::__init__::225::IOProcess::(_processLogs) (181) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:06,210::__init__::225::IOProcess::(_processLogs) (181) Building response >Thread-17::DEBUG::2014-08-22 09:20:06,211::__init__::225::IOProcess::(_processLogs) (181) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:06,211::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:06,211::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:20:07,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:07,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119831 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:09,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:09,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204623 s, 50.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:11,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a20cc78-42cc-45e1-93f8-f31e8e9ffa07`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:11,877::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:20:11,877::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219096', 'lastCheck': '5.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000211333', 'lastCheck': '5.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:11,877::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a20cc78-42cc-45e1-93f8-f31e8e9ffa07`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219096', 'lastCheck': '5.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000211333', 'lastCheck': '5.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:11,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a20cc78-42cc-45e1-93f8-f31e8e9ffa07`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:11,877::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:11,877::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:11,877::task::993::Storage.TaskManager.Task::(_decref) Task=`4a20cc78-42cc-45e1-93f8-f31e8e9ffa07`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:11,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:12,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120555 s, 84.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:12,925::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:20:12,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`3eeab0c5-2698-4b52-9a55-08febba7eeaf`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:12,926::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:20:12,927::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:12,927::task::1191::Storage.TaskManager.Task::(prepare) Task=`3eeab0c5-2698-4b52-9a55-08febba7eeaf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:12,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`3eeab0c5-2698-4b52-9a55-08febba7eeaf`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:12,927::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:12,927::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:12,927::task::993::Storage.TaskManager.Task::(_decref) Task=`3eeab0c5-2698-4b52-9a55-08febba7eeaf`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:20:12,958::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:20:12,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9135fca-13fa-431c-9bcc-66a0a6d286c9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:20:12,958::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:20:12,959::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fb087794-53d2-462c-a6fd-3c358bcda7a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:20:12,959::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:20:12,959::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:20:12,959::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fb087794-53d2-462c-a6fd-3c358bcda7a1`::Granted request >Thread-46::DEBUG::2014-08-22 09:20:12,959::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c9135fca-13fa-431c-9bcc-66a0a6d286c9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:20:12,959::task::993::Storage.TaskManager.Task::(_decref) Task=`c9135fca-13fa-431c-9bcc-66a0a6d286c9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:20:12,961::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:12,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9135fca-13fa-431c-9bcc-66a0a6d286c9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:12,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9135fca-13fa-431c-9bcc-66a0a6d286c9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:20:12,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:20:12,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:20:12,962::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:20:12,962::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:20:12,962::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:20:12,962::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:20:12,962::task::993::Storage.TaskManager.Task::(_decref) Task=`c9135fca-13fa-431c-9bcc-66a0a6d286c9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:14,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:14,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116539 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:16,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:16,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011625 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:16,157::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:16,157::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:16,158::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,158::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,158::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,158::__init__::225::IOProcess::(_processLogs) (182) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:16,158::__init__::225::IOProcess::(_processLogs) (182) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:16,158::__init__::225::IOProcess::(_processLogs) (182) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:16,159::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:20:16,159::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:16,160::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:20:16,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000190818 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:16,166::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:16,167::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:16,167::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,167::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,167::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,167::__init__::225::IOProcess::(_processLogs) (183) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:16,167::__init__::225::IOProcess::(_processLogs) (183) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:16,168::__init__::225::IOProcess::(_processLogs) (183) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:16,168::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:16,169::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) (184) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:16,212::__init__::225::IOProcess::(_processLogs) (184) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:16,213::__init__::225::IOProcess::(_processLogs) (184) Queuing response >Thread-22::DEBUG::2014-08-22 09:20:16,213::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:16,213::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:16,218::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:20:16,219::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000247 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:16,219::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:16,220::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:16,220::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,220::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,220::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,220::__init__::225::IOProcess::(_processLogs) (185) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:16,220::__init__::225::IOProcess::(_processLogs) (185) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:16,221::__init__::225::IOProcess::(_processLogs) (185) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,222::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,223::__init__::225::IOProcess::(_processLogs) (186) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:16,223::__init__::225::IOProcess::(_processLogs) (186) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:16,223::__init__::225::IOProcess::(_processLogs) (186) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:16,223::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:16,223::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:16,224::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:20:16,224::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:16,224::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,224::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) (187) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) (187) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) (187) Building response >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) (187) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:20:16,225::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:16,226::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:16,226::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:16,226::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:16,226::__init__::225::IOProcess::(_processLogs) (188) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:16,227::__init__::225::IOProcess::(_processLogs) (188) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:16,227::__init__::225::IOProcess::(_processLogs) (188) Building response >Thread-17::DEBUG::2014-08-22 09:20:16,227::__init__::225::IOProcess::(_processLogs) (188) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:16,227::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:16,227::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:20:18,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:18,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118883 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:20,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:20,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115699 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:22,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:22,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117055 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:23,054::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:20:23,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`cae87abd-dadb-442b-b5c4-538b908caf85`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:23,055::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:20:23,055::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:23,055::task::1191::Storage.TaskManager.Task::(prepare) Task=`cae87abd-dadb-442b-b5c4-538b908caf85`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:23,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`cae87abd-dadb-442b-b5c4-538b908caf85`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:23,055::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:23,055::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:23,055::task::993::Storage.TaskManager.Task::(_decref) Task=`cae87abd-dadb-442b-b5c4-538b908caf85`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:20:23,079::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:20:23,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8207151-7bd5-4cf7-9506-69c643f0fe2f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:20:23,079::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:20:23,079::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`02286152-0b69-46cf-8f43-c02a93ed91fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:20:23,079::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:20:23,080::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:20:23,080::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`02286152-0b69-46cf-8f43-c02a93ed91fb`::Granted request >Thread-46::DEBUG::2014-08-22 09:20:23,080::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b8207151-7bd5-4cf7-9506-69c643f0fe2f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:20:23,080::task::993::Storage.TaskManager.Task::(_decref) Task=`b8207151-7bd5-4cf7-9506-69c643f0fe2f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:20:23,082::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:23,082::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8207151-7bd5-4cf7-9506-69c643f0fe2f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:23,082::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8207151-7bd5-4cf7-9506-69c643f0fe2f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:20:23,082::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:20:23,082::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:20:23,082::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:20:23,083::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:20:23,083::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:20:23,083::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:20:23,083::task::993::Storage.TaskManager.Task::(_decref) Task=`b8207151-7bd5-4cf7-9506-69c643f0fe2f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:24,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:24,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115432 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:26,169::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:26,169::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:26,169::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,169::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,169::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,170::__init__::225::IOProcess::(_processLogs) (189) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:26,170::__init__::225::IOProcess::(_processLogs) (189) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:26,170::__init__::225::IOProcess::(_processLogs) (189) Queuing response >Thread-21::DEBUG::2014-08-22 09:20:26,171::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:26,171::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,177::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:20:26,178::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000191952 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:26,178::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:26,179::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:26,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,179::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,179::__init__::225::IOProcess::(_processLogs) (190) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:26,179::__init__::225::IOProcess::(_processLogs) (190) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:26,180::__init__::225::IOProcess::(_processLogs) (190) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:26,180::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,181::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) (191) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:26,227::__init__::225::IOProcess::(_processLogs) (191) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:26,228::__init__::225::IOProcess::(_processLogs) (191) Queuing response >Thread-22::DEBUG::2014-08-22 09:20:26,228::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:26,229::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,234::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:20:26,235::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000249892 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) (192) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:26,236::__init__::225::IOProcess::(_processLogs) (192) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:26,237::__init__::225::IOProcess::(_processLogs) (192) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:26,238::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,238::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:26,238::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:20:26,238::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:26,238::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) (193) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) (193) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) (193) Building response >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) (193) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:20:26,239::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:26,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,240::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,240::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,240::__init__::225::IOProcess::(_processLogs) (194) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:26,240::__init__::225::IOProcess::(_processLogs) (194) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:26,240::__init__::225::IOProcess::(_processLogs) (194) Building response >Thread-17::DEBUG::2014-08-22 09:20:26,241::__init__::225::IOProcess::(_processLogs) (194) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:26,241::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,241::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:26,241::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:20:26,241::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:26,241::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) (195) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) (195) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) (195) Building response >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) (195) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:26,242::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:26,243::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:20:26,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:26,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121658 s, 84.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:27,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed362b41-b9db-4d22-849e-a9f32fa556cd`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:27,110::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:20:27,110::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000191952', 'lastCheck': '0.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000249892', 'lastCheck': '0.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:27,110::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed362b41-b9db-4d22-849e-a9f32fa556cd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000191952', 'lastCheck': '0.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000249892', 'lastCheck': '0.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:27,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed362b41-b9db-4d22-849e-a9f32fa556cd`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:27,110::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:27,110::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:27,111::task::993::Storage.TaskManager.Task::(_decref) Task=`ed362b41-b9db-4d22-849e-a9f32fa556cd`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:28,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:28,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116975 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:30,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:30,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116106 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:32,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:32,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179947 s, 56.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:33,132::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:20:33,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`7032c2de-4dae-47e9-b5bd-9fef8b6f43a8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:33,132::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:20:33,133::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:33,133::task::1191::Storage.TaskManager.Task::(prepare) Task=`7032c2de-4dae-47e9-b5bd-9fef8b6f43a8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:33,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`7032c2de-4dae-47e9-b5bd-9fef8b6f43a8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:33,133::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:33,133::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:33,133::task::993::Storage.TaskManager.Task::(_decref) Task=`7032c2de-4dae-47e9-b5bd-9fef8b6f43a8`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:20:33,189::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:20:33,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccd38395-0ee4-4d82-a0e3-403032c58ab5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:20:33,189::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:20:33,189::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d0fdcb29-d46b-4202-aaea-0f8b0ca2e628`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:20:33,189::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:20:33,190::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:20:33,190::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d0fdcb29-d46b-4202-aaea-0f8b0ca2e628`::Granted request >Thread-46::DEBUG::2014-08-22 09:20:33,190::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ccd38395-0ee4-4d82-a0e3-403032c58ab5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:20:33,190::task::993::Storage.TaskManager.Task::(_decref) Task=`ccd38395-0ee4-4d82-a0e3-403032c58ab5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:20:33,192::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:33,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccd38395-0ee4-4d82-a0e3-403032c58ab5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:33,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccd38395-0ee4-4d82-a0e3-403032c58ab5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:20:33,192::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:20:33,192::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:20:33,192::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:20:33,192::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:20:33,193::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:20:33,193::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:20:33,193::task::993::Storage.TaskManager.Task::(_decref) Task=`ccd38395-0ee4-4d82-a0e3-403032c58ab5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:34,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:34,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116817 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,181::__init__::225::IOProcess::(_processLogs) (196) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:36,182::__init__::225::IOProcess::(_processLogs) (196) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:36,182::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(196) Building response >Thread-21::DEBUG::2014-08-22 09:20:36,183::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:36,183::__init__::225::IOProcess::(_processLogs) (196) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,189::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:20:36,190::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178641 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:36,190::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:36,191::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:36,191::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:36,191::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,191::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,191::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,192::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,192::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,192::__init__::225::IOProcess::(_processLogs) (197) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:36,192::__init__::225::IOProcess::(_processLogs) (197) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:36,192::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(197) Building response >Thread-17::DEBUG::2014-08-22 09:20:36,193::__init__::225::IOProcess::(_processLogs) (197) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,193::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:36,193::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:36,242::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) (198) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) (198) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) (198) Building response >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) (198) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,243::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:36,244::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:20:36,244::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:20:36,250::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000233616 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:36,250::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) (199) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) (199) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) (199) Building response >Thread-17::DEBUG::2014-08-22 09:20:36,251::__init__::225::IOProcess::(_processLogs) (199) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,252::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:20:36,253::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:36,253::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:36,253::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:20:36,253::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,253::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) (200) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) (200) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) (200) Building response >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) (200) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:36,254::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:36,255::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:36,255::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:20:36,255::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,255::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,255::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) (201) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) (201) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) (201) Building response >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) (201) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:36,256::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:20:36,257::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:36,257::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:36,257::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) (202) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) (202) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) (202) Building response >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) (202) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:36,258::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:20:36,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:36,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123462 s, 82.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:38,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:38,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116831 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:40,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:40,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118296 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:42,332::task::595::Storage.TaskManager.Task::(_updateState) Task=`8925f778-ee90-4240-955a-4e414f4c0b57`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:42,332::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:20:42,332::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178641', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000233616', 'lastCheck': '6.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:42,332::task::1191::Storage.TaskManager.Task::(prepare) Task=`8925f778-ee90-4240-955a-4e414f4c0b57`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178641', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000233616', 'lastCheck': '6.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:42,332::task::595::Storage.TaskManager.Task::(_updateState) Task=`8925f778-ee90-4240-955a-4e414f4c0b57`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:42,332::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:42,332::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:42,333::task::993::Storage.TaskManager.Task::(_decref) Task=`8925f778-ee90-4240-955a-4e414f4c0b57`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:42,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:42,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117312 s, 87.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:43,245::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:20:43,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fe8169c-c8b0-48e8-9697-56878372ac62`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:43,246::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:20:43,246::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:43,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`4fe8169c-c8b0-48e8-9697-56878372ac62`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:43,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fe8169c-c8b0-48e8-9697-56878372ac62`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:43,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:43,246::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:43,246::task::993::Storage.TaskManager.Task::(_decref) Task=`4fe8169c-c8b0-48e8-9697-56878372ac62`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:20:43,287::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:20:43,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`87ae7700-8de3-4d3a-8207-1ba4e429e9c1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:20:43,287::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:20:43,288::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d67af56c-d1e9-4abf-bcd0-0c6b8a538446`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:20:43,288::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:20:43,288::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:20:43,288::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d67af56c-d1e9-4abf-bcd0-0c6b8a538446`::Granted request >Thread-46::DEBUG::2014-08-22 09:20:43,288::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`87ae7700-8de3-4d3a-8207-1ba4e429e9c1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:20:43,288::task::993::Storage.TaskManager.Task::(_decref) Task=`87ae7700-8de3-4d3a-8207-1ba4e429e9c1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:20:43,290::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:43,290::task::1191::Storage.TaskManager.Task::(prepare) Task=`87ae7700-8de3-4d3a-8207-1ba4e429e9c1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:43,290::task::595::Storage.TaskManager.Task::(_updateState) Task=`87ae7700-8de3-4d3a-8207-1ba4e429e9c1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:20:43,290::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:20:43,290::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:20:43,291::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:20:43,291::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:20:43,291::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:20:43,291::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:20:43,291::task::993::Storage.TaskManager.Task::(_decref) Task=`87ae7700-8de3-4d3a-8207-1ba4e429e9c1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:44,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:44,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116445 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) (203) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:46,194::__init__::225::IOProcess::(_processLogs) (203) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:46,195::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(203) Building response >Thread-21::DEBUG::2014-08-22 09:20:46,196::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:46,196::__init__::225::IOProcess::(_processLogs) (203) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,202::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:20:46,203::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000184449 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:46,203::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:46,204::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,204::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:46,204::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,204::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:46,204::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,205::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:46,205::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,205::__init__::225::IOProcess::(_processLogs) (204) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:46,205::__init__::225::IOProcess::(_processLogs) (204) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:46,205::__init__::225::IOProcess::(_processLogs) (204) Building response >Thread-17::DEBUG::2014-08-22 09:20:46,206::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(204) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,206::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:46,206::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:46,258::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,258::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:46,258::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,259::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:20:46,259::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:46,259::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,265::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:20:46,266::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000210507 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:46,266::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,267::__init__::225::IOProcess::(_processLogs) (205) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:46,267::__init__::225::IOProcess::(_processLogs) (205) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:46,267::__init__::225::IOProcess::(_processLogs) (205) Building response >Thread-17::DEBUG::2014-08-22 09:20:46,267::__init__::225::IOProcess::(_processLogs) (205) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,267::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) (206) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:46,268::__init__::225::IOProcess::(_processLogs) (206) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:46,269::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(206) Building response >Thread-17::DEBUG::2014-08-22 09:20:46,270::__init__::225::IOProcess::(_processLogs) (206) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,270::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:46,270::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:46,270::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,270::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:20:46,270::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) (207) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) (207) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:46,271::__init__::225::IOProcess::(_processLogs) (207) Building response >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) (207) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,272::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) (208) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) (208) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) (208) Building response >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) (208) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:46,273::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:46,274::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:46,274::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:20:46,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:46,274::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:46,274::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) (209) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) (209) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) (209) Building response >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) (209) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:46,275::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:20:46,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:46,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120308 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:48,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:49,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118439 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:51,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:51,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116479 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:20:53,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:53,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011506 s, 89.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:53,347::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:20:53,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3725f7f-1789-4884-be86-d6f79464e6af`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:53,348::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:20:53,348::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:53,348::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3725f7f-1789-4884-be86-d6f79464e6af`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:20:53,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3725f7f-1789-4884-be86-d6f79464e6af`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:53,348::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:53,348::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:53,348::task::993::Storage.TaskManager.Task::(_decref) Task=`c3725f7f-1789-4884-be86-d6f79464e6af`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:20:53,387::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:20:53,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d310802-d770-4108-a67e-b84984f420f1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:20:53,388::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:20:53,388::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`167dc68b-743c-4fad-8d58-7f98f1b6b4cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:20:53,388::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:20:53,388::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:20:53,388::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`167dc68b-743c-4fad-8d58-7f98f1b6b4cd`::Granted request >Thread-46::DEBUG::2014-08-22 09:20:53,388::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1d310802-d770-4108-a67e-b84984f420f1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:20:53,389::task::993::Storage.TaskManager.Task::(_decref) Task=`1d310802-d770-4108-a67e-b84984f420f1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:20:53,390::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:53,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d310802-d770-4108-a67e-b84984f420f1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:20:53,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d310802-d770-4108-a67e-b84984f420f1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:20:53,391::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:20:53,391::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:20:53,391::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:20:53,391::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:20:53,391::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:20:53,391::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:20:53,391::task::993::Storage.TaskManager.Task::(_decref) Task=`1d310802-d770-4108-a67e-b84984f420f1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:55,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:55,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116191 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:56,207::__init__::225::IOProcess::(_processLogs) (210) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:56,208::__init__::225::IOProcess::(_processLogs) (210) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:56,208::__init__::225::IOProcess::(_processLogs) DEBUG|(210) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,209::__init__::225::IOProcess::(_processLogs) (210) Queuing response >Thread-21::DEBUG::2014-08-22 09:20:56,209::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:56,210::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,216::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:20:56,216::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000202443 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,217::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:56,218::__init__::225::IOProcess::(_processLogs) (211) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:56,218::__init__::225::IOProcess::(_processLogs) (211) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:56,218::__init__::225::IOProcess::(_processLogs) DEBUG|(211) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,219::__init__::225::IOProcess::(_processLogs) (211) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:56,219::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,219::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:56,275::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,275::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:56,275::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,276::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,276::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:20:56,276::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:20:56,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,283::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:20:56,283::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021985 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:20:56,284::__init__::225::IOProcess::(_processLogs) (212) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:56,284::__init__::225::IOProcess::(_processLogs) (212) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:56,284::__init__::225::IOProcess::(_processLogs) (212) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,284::__init__::225::IOProcess::(_processLogs) (212) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,285::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:56,286::__init__::225::IOProcess::(_processLogs) (213) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:20:56,286::__init__::225::IOProcess::(_processLogs) (213) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:20:56,286::__init__::225::IOProcess::(_processLogs) DEBUG|(213) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,287::__init__::225::IOProcess::(_processLogs) (213) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:56,287::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,287::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:20:56,287::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,287::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:20:56,287::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) (214) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) (214) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) (214) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,288::__init__::225::IOProcess::(_processLogs) (214) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:56,289::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,289::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:56,289::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,289::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:20:56,289::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,289::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) (215) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) (215) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) (215) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) (215) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,290::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:20:56,291::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:20:56,291::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:20:56,291::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:20:56,291::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:20:56,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) (216) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) (216) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) (216) Building response >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) (216) Queuing response >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:20:56,292::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:20:57,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:57,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117979 s, 86.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:20:57,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`27bb1f55-bd0d-46c2-8667-c5c6313efafa`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:20:57,559::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:20:57,559::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000202443', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021985', 'lastCheck': '1.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:57,559::task::1191::Storage.TaskManager.Task::(prepare) Task=`27bb1f55-bd0d-46c2-8667-c5c6313efafa`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000202443', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021985', 'lastCheck': '1.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:20:57,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`27bb1f55-bd0d-46c2-8667-c5c6313efafa`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:20:57,559::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:20:57,560::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:20:57,560::task::993::Storage.TaskManager.Task::(_decref) Task=`27bb1f55-bd0d-46c2-8667-c5c6313efafa`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:20:59,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:20:59,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123361 s, 83.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:01,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:01,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118187 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:03,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:03,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011364 s, 90.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:03,436::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:21:03,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac549de8-0547-41a3-aae6-bd567a670b75`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:03,440::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:21:03,440::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:03,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac549de8-0547-41a3-aae6-bd567a670b75`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:03,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac549de8-0547-41a3-aae6-bd567a670b75`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:03,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:03,441::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:03,441::task::993::Storage.TaskManager.Task::(_decref) Task=`ac549de8-0547-41a3-aae6-bd567a670b75`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:21:03,480::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:21:03,481::task::595::Storage.TaskManager.Task::(_updateState) Task=`2471b594-49c6-4065-90bb-926d68f1d36a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:21:03,481::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:21:03,481::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d4b7b286-8991-40cf-9235-55e9595ee45b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:21:03,481::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:21:03,481::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:21:03,481::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d4b7b286-8991-40cf-9235-55e9595ee45b`::Granted request >Thread-46::DEBUG::2014-08-22 09:21:03,481::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2471b594-49c6-4065-90bb-926d68f1d36a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:21:03,482::task::993::Storage.TaskManager.Task::(_decref) Task=`2471b594-49c6-4065-90bb-926d68f1d36a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:21:03,485::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:03,485::task::1191::Storage.TaskManager.Task::(prepare) Task=`2471b594-49c6-4065-90bb-926d68f1d36a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:03,485::task::595::Storage.TaskManager.Task::(_updateState) Task=`2471b594-49c6-4065-90bb-926d68f1d36a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:21:03,485::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:21:03,485::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:21:03,485::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:21:03,485::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:21:03,486::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:21:03,486::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:21:03,486::task::993::Storage.TaskManager.Task::(_decref) Task=`2471b594-49c6-4065-90bb-926d68f1d36a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:05,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:05,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118236 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) (217) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:06,220::__init__::225::IOProcess::(_processLogs) (217) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:06,221::__init__::225::IOProcess::(_processLogs) DEBUG|(217) Building response >Thread-21::DEBUG::2014-08-22 09:21:06,222::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:06,222::__init__::225::IOProcess::(_processLogs) (217) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:06,228::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:21:06,229::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000218256 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:06,229::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:06,230::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,230::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:06,230::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,230::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,230::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,231::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,231::__init__::225::IOProcess::(_processLogs) (218) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:06,231::__init__::225::IOProcess::(_processLogs) (218) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:06,231::__init__::225::IOProcess::(_processLogs) DEBUG|(218) Building response >Thread-17::DEBUG::2014-08-22 09:21:06,232::__init__::225::IOProcess::(_processLogs) (218) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:06,232::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:06,232::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:06,292::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,292::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:06,292::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,292::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,292::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,293::__init__::225::IOProcess::(_processLogs) (219) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:06,293::__init__::225::IOProcess::(_processLogs) (219) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:06,293::__init__::225::IOProcess::(_processLogs) DEBUG|(219) Building response >Thread-17::DEBUG::2014-08-22 09:21:06,294::__init__::225::IOProcess::(_processLogs) (219) Queuing response >Thread-22::DEBUG::2014-08-22 09:21:06,294::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:06,294::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:06,300::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:21:06,301::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021417 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:06,301::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,301::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) (220) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:06,302::__init__::225::IOProcess::(_processLogs) (220) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:06,303::__init__::225::IOProcess::(_processLogs) DEBUG|(220) Building response >Thread-17::DEBUG::2014-08-22 09:21:06,303::__init__::225::IOProcess::(_processLogs) (220) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:06,303::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:06,303::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:06,304::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,304::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:21:06,304::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,304::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) (221) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) (221) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) (221) Building response >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) (221) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:06,305::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,306::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:21:06,306::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,306::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,306::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,306::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) (222) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) (222) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) (222) Building response >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) (222) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:21:06,307::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) (223) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) (223) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:06,308::__init__::225::IOProcess::(_processLogs) (223) Building response >Thread-17::DEBUG::2014-08-22 09:21:06,309::__init__::225::IOProcess::(_processLogs) (223) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:06,309::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:06,309::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:21:07,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:07,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138351 s, 74.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:09,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:09,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117101 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:11,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:11,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116127 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:12,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0b2b9a6-af51-4111-aa3d-4eaa70341612`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:12,750::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:21:12,750::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000218256', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021417', 'lastCheck': '6.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:12,750::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0b2b9a6-af51-4111-aa3d-4eaa70341612`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000218256', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021417', 'lastCheck': '6.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:12,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0b2b9a6-af51-4111-aa3d-4eaa70341612`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:12,750::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:12,750::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:12,750::task::993::Storage.TaskManager.Task::(_decref) Task=`d0b2b9a6-af51-4111-aa3d-4eaa70341612`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:21:13,531::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:21:13,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`0983ee7b-ac64-47ff-9b4f-ae00c65095e9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:13,532::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:21:13,532::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:13,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`0983ee7b-ac64-47ff-9b4f-ae00c65095e9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:13,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`0983ee7b-ac64-47ff-9b4f-ae00c65095e9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:13,532::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:13,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:13,532::task::993::Storage.TaskManager.Task::(_decref) Task=`0983ee7b-ac64-47ff-9b4f-ae00c65095e9`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:21:13,572::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:21:13,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4137696-d630-491d-bd87-538dacba2147`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:21:13,573::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:21:13,573::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`da8991de-9c4c-41ac-8db2-08df54674a66`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:21:13,573::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:21:13,573::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:21:13,573::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`da8991de-9c4c-41ac-8db2-08df54674a66`::Granted request >Thread-46::DEBUG::2014-08-22 09:21:13,574::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e4137696-d630-491d-bd87-538dacba2147`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:21:13,574::task::993::Storage.TaskManager.Task::(_decref) Task=`e4137696-d630-491d-bd87-538dacba2147`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:21:13,576::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:13,576::task::1191::Storage.TaskManager.Task::(prepare) Task=`e4137696-d630-491d-bd87-538dacba2147`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:13,576::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4137696-d630-491d-bd87-538dacba2147`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:21:13,576::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:21:13,576::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:21:13,576::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:21:13,576::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:21:13,576::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:21:13,576::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:21:13,576::task::993::Storage.TaskManager.Task::(_decref) Task=`e4137696-d630-491d-bd87-538dacba2147`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:13,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:13,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116755 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:15,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:15,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117098 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) (224) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:16,233::__init__::225::IOProcess::(_processLogs) (224) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:16,234::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(224) Building response >Thread-21::DEBUG::2014-08-22 09:21:16,235::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:16,235::__init__::225::IOProcess::(_processLogs) (224) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,241::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:16,241::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:21:16,242::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000215152 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) (225) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:16,243::__init__::225::IOProcess::(_processLogs) (225) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:16,244::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(225) Building response >Thread-17::DEBUG::2014-08-22 09:21:16,245::__init__::225::IOProcess::(_processLogs) (225) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:16,245::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:16,313::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,313::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:16,313::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,313::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,314::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,314::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,314::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,314::__init__::225::IOProcess::(_processLogs) (226) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:16,314::__init__::225::IOProcess::(_processLogs) (226) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:16,314::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(226) Building response >Thread-22::DEBUG::2014-08-22 09:21:16,315::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:16,315::__init__::225::IOProcess::(_processLogs) (226) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,321::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:21:16,322::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000192936 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:16,322::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:16,323::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,323::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) (227) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:16,324::__init__::225::IOProcess::(_processLogs) (227) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:16,325::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(227) Building response >Thread-17::DEBUG::2014-08-22 09:21:16,325::__init__::225::IOProcess::(_processLogs) (227) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,325::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:16,326::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:16,326::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,326::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:21:16,326::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,326::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) (228) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) (228) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) (228) Building response >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) (228) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:16,327::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,328::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:21:16,328::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,328::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,328::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,328::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,328::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) (229) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) (229) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) (229) Building response >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) (229) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:21:16,329::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:16,330::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:16,330::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:16,330::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) (230) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) (230) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) (230) Building response >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) (230) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:16,331::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:21:17,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:17,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119383 s, 85.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:19,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:19,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116398 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:21,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:21,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011695 s, 87.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:23,626::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:21:23,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf52ea52-fd0e-49c8-ab81-4086105afd68`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:23,626::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:21:23,626::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:23,626::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf52ea52-fd0e-49c8-ab81-4086105afd68`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:23,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf52ea52-fd0e-49c8-ab81-4086105afd68`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:23,627::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:23,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:23,627::task::993::Storage.TaskManager.Task::(_decref) Task=`cf52ea52-fd0e-49c8-ab81-4086105afd68`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:21:23,652::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:21:23,652::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a963ae0-defe-46fc-b4a8-a2a408f12d90`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:21:23,652::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:21:23,652::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`893e001b-167c-49da-8c01-6dfd3a09cec0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:21:23,652::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:21:23,652::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:21:23,653::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`893e001b-167c-49da-8c01-6dfd3a09cec0`::Granted request >Thread-46::DEBUG::2014-08-22 09:21:23,653::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1a963ae0-defe-46fc-b4a8-a2a408f12d90`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:21:23,653::task::993::Storage.TaskManager.Task::(_decref) Task=`1a963ae0-defe-46fc-b4a8-a2a408f12d90`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:21:23,655::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:23,655::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a963ae0-defe-46fc-b4a8-a2a408f12d90`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:23,655::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a963ae0-defe-46fc-b4a8-a2a408f12d90`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:21:23,655::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:21:23,655::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:21:23,655::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:21:23,655::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:21:23,655::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:21:23,655::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:21:23,656::task::993::Storage.TaskManager.Task::(_decref) Task=`1a963ae0-defe-46fc-b4a8-a2a408f12d90`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:23,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:23,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120184 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:25,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:25,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117262 s, 87.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:26,245::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,245::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:26,245::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,246::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,246::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,246::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,246::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,246::__init__::225::IOProcess::(_processLogs) (231) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:26,246::__init__::225::IOProcess::(_processLogs) (231) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:26,247::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(231) Building response >Thread-21::DEBUG::2014-08-22 09:21:26,247::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:26,247::__init__::225::IOProcess::(_processLogs) (231) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,253::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:21:26,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000226901 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:26,255::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) (232) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:26,256::__init__::225::IOProcess::(_processLogs) (232) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:26,257::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(232) Building response >Thread-17::DEBUG::2014-08-22 09:21:26,258::__init__::225::IOProcess::(_processLogs) (232) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,258::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:26,258::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,331::__init__::225::IOProcess::(_processLogs) (233) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:26,332::__init__::225::IOProcess::(_processLogs) (233) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:26,332::__init__::225::IOProcess::(_processLogs) (233) Building response >Thread-17::DEBUG::2014-08-22 09:21:26,332::__init__::225::IOProcess::(_processLogs) (233) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,332::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:26,332::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:21:26,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:21:26,338::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000247382 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:26,339::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,339::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:26,339::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,340::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,340::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,340::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,340::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) (234) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) (234) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) (234) Building response >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) (234) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:21:26,341::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,342::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) (235) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) (235) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) (235) Building response >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) (235) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:26,343::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:26,344::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,344::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:21:26,344::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,344::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,344::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,344::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) (236) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) (236) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) (236) Building response >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) (236) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:21:26,345::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:26,346::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:26,346::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:26,346::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) (237) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) (237) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) (237) Building response >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) (237) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:26,347::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:21:27,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`263373d1-043b-411b-9ebf-ad1a81973b78`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:27,925::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:21:27,925::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000226901', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000247382', 'lastCheck': '1.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:27,925::task::1191::Storage.TaskManager.Task::(prepare) Task=`263373d1-043b-411b-9ebf-ad1a81973b78`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000226901', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000247382', 'lastCheck': '1.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:27,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`263373d1-043b-411b-9ebf-ad1a81973b78`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:27,925::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:27,925::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:27,925::task::993::Storage.TaskManager.Task::(_decref) Task=`263373d1-043b-411b-9ebf-ad1a81973b78`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:27,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:27,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161727 s, 63.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:29,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:30,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118617 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:32,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:32,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233781 s, 43.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:33,706::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:21:33,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdd301a3-a9eb-434b-be4f-b5c6c6b2341e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:33,706::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:21:33,706::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:33,707::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdd301a3-a9eb-434b-be4f-b5c6c6b2341e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:33,707::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdd301a3-a9eb-434b-be4f-b5c6c6b2341e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:33,707::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:33,707::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:33,707::task::993::Storage.TaskManager.Task::(_decref) Task=`bdd301a3-a9eb-434b-be4f-b5c6c6b2341e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:21:33,737::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:21:33,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`d17a670d-adb3-403b-b174-52dad01ae191`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:21:33,738::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:21:33,738::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`00144c7d-4cd5-4a37-8102-e3981aa4cbbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:21:33,738::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:21:33,738::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:21:33,738::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`00144c7d-4cd5-4a37-8102-e3981aa4cbbd`::Granted request >Thread-46::DEBUG::2014-08-22 09:21:33,739::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d17a670d-adb3-403b-b174-52dad01ae191`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:21:33,739::task::993::Storage.TaskManager.Task::(_decref) Task=`d17a670d-adb3-403b-b174-52dad01ae191`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:21:33,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:33,741::task::1191::Storage.TaskManager.Task::(prepare) Task=`d17a670d-adb3-403b-b174-52dad01ae191`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:33,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`d17a670d-adb3-403b-b174-52dad01ae191`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:21:33,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:21:33,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:21:33,741::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:21:33,741::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:21:33,741::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:21:33,741::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:21:33,741::task::993::Storage.TaskManager.Task::(_decref) Task=`d17a670d-adb3-403b-b174-52dad01ae191`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:34,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:34,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114476 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:36,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:36,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115451 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:36,259::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,260::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:36,260::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,261::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:36,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:36,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:36,265::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-21::DEBUG::2014-08-22 09:21:36,266::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:36,266::__init__::225::IOProcess::(_processLogs) (238) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:36,272::__init__::225::IOProcess::(_processLogs) (238) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:36,273::__init__::225::IOProcess::(_processLogs) (238) Building response >Thread-21::DEBUG::2014-08-22 09:21:36,274::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182968 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:36,274::__init__::225::IOProcess::(_processLogs) (238) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,276::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,276::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:36,277::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,277::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:36,279::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,280::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:36,281::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:36,281::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:36,281::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:36,282::__init__::225::IOProcess::(_processLogs) (239) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:36,282::__init__::225::IOProcess::(_processLogs) (239) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:36,283::__init__::225::IOProcess::(_processLogs) (239) Building response >Thread-17::DEBUG::2014-08-22 09:21:36,283::__init__::225::IOProcess::(_processLogs) (239) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,284::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,285::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:36,347::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,347::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:36,347::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,347::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:21:36,348::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:36,348::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:36,354::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:21:36,355::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000217926 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:36,355::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) (240) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) (240) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) (240) Building response >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) (240) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:36,356::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) (241) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:36,357::__init__::225::IOProcess::(_processLogs) (241) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:36,358::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(241) Building response >Thread-17::DEBUG::2014-08-22 09:21:36,358::__init__::225::IOProcess::(_processLogs) (241) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,358::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,359::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:36,359::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,359::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:21:36,359::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,359::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) (242) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) (242) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) (242) Building response >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) (242) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:36,360::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,361::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:21:36,361::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,361::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:36,361::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:36,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) (243) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) (243) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) (243) Building response >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) (243) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:36,362::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:21:36,363::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:36,363::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:36,363::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) (244) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) (244) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) (244) Building response >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) (244) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:36,364::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:21:38,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:38,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118931 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:40,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:40,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116245 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:42,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:42,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118375 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:43,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba1a654a-1e4b-42b9-8083-ce60b93c067c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:43,131::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:21:43,131::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182968', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000217926', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:43,131::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba1a654a-1e4b-42b9-8083-ce60b93c067c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182968', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000217926', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:43,131::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba1a654a-1e4b-42b9-8083-ce60b93c067c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:43,131::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:43,131::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:43,131::task::993::Storage.TaskManager.Task::(_decref) Task=`ba1a654a-1e4b-42b9-8083-ce60b93c067c`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:21:43,875::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:21:43,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`06bbdf5b-239e-4e36-84cf-687512174f10`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:43,876::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:21:43,876::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:43,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`06bbdf5b-239e-4e36-84cf-687512174f10`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:43,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`06bbdf5b-239e-4e36-84cf-687512174f10`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:43,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:43,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:43,876::task::993::Storage.TaskManager.Task::(_decref) Task=`06bbdf5b-239e-4e36-84cf-687512174f10`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:21:43,908::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:21:43,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`becaa2fb-7c6a-4ff0-97fe-69a23918e064`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:21:43,908::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:21:43,908::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`38ba440d-6985-4590-b21c-e11e3b130cc1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:21:43,909::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:21:43,909::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:21:43,909::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`38ba440d-6985-4590-b21c-e11e3b130cc1`::Granted request >Thread-46::DEBUG::2014-08-22 09:21:43,909::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`becaa2fb-7c6a-4ff0-97fe-69a23918e064`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:21:43,909::task::993::Storage.TaskManager.Task::(_decref) Task=`becaa2fb-7c6a-4ff0-97fe-69a23918e064`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:21:43,911::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:43,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`becaa2fb-7c6a-4ff0-97fe-69a23918e064`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:43,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`becaa2fb-7c6a-4ff0-97fe-69a23918e064`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:21:43,911::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:21:43,911::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:21:43,912::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:21:43,912::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:21:43,912::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:21:43,912::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:21:43,912::task::993::Storage.TaskManager.Task::(_decref) Task=`becaa2fb-7c6a-4ff0-97fe-69a23918e064`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:44,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:44,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114739 s, 89.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:46,281::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,281::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:46,281::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,281::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,281::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:46,282::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,282::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:46,282::__init__::225::IOProcess::(_processLogs) (245) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:46,282::__init__::225::IOProcess::(_processLogs) (245) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:46,282::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(245) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,283::__init__::225::IOProcess::(_processLogs) (245) Queuing response >Thread-21::DEBUG::2014-08-22 09:21:46,283::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:46,284::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,290::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:21:46,292::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000203074 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:46,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,294::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:46,294::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,294::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,294::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:46,295::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) (246) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) (246) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) (246) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) (246) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,296::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:46,364::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,364::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:46,364::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,365::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,365::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:21:46,365::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:46,366::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,372::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:21:46,372::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200407 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:46,373::__init__::225::IOProcess::(_processLogs) (247) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:46,373::__init__::225::IOProcess::(_processLogs) (247) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:46,373::__init__::225::IOProcess::(_processLogs) (247) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) (247) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:46,374::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,375::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:46,375::__init__::225::IOProcess::(_processLogs) (248) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:46,375::__init__::225::IOProcess::(_processLogs) (248) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:46,375::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(248) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,376::__init__::225::IOProcess::(_processLogs) (248) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:46,376::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,376::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:46,376::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,376::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:21:46,376::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,377::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,377::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:46,377::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,377::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:46,377::__init__::225::IOProcess::(_processLogs) (249) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:46,377::__init__::225::IOProcess::(_processLogs) (249) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) (249) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) (249) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,378::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) (250) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) (250) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) (250) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) (250) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,379::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:46,380::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:46,380::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:21:46,380::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:46,380::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:46,380::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) (251) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) (251) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) (251) Building response >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) (251) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:46,381::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:21:46,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:46,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115497 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:48,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:48,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118403 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:50,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:50,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121376 s, 84.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:21:52,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:52,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011545 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:53,979::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:21:53,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4589b23-c1e4-4577-b74a-eba0a26380b6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:53,980::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:21:53,980::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:53,980::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4589b23-c1e4-4577-b74a-eba0a26380b6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:21:53,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4589b23-c1e4-4577-b74a-eba0a26380b6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:53,980::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:53,980::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:53,980::task::993::Storage.TaskManager.Task::(_decref) Task=`d4589b23-c1e4-4577-b74a-eba0a26380b6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:21:54,012::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:21:54,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`583a4fe8-0944-4e27-b680-df272cf147ad`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:21:54,012::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:21:54,013::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cdf4d972-03af-4ab6-81dd-2ae77c2162c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:21:54,013::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:21:54,013::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:21:54,013::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cdf4d972-03af-4ab6-81dd-2ae77c2162c9`::Granted request >Thread-46::DEBUG::2014-08-22 09:21:54,013::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`583a4fe8-0944-4e27-b680-df272cf147ad`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:21:54,013::task::993::Storage.TaskManager.Task::(_decref) Task=`583a4fe8-0944-4e27-b680-df272cf147ad`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:21:54,015::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:54,015::task::1191::Storage.TaskManager.Task::(prepare) Task=`583a4fe8-0944-4e27-b680-df272cf147ad`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:21:54,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`583a4fe8-0944-4e27-b680-df272cf147ad`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:21:54,015::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:21:54,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:21:54,016::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:21:54,016::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:21:54,016::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:21:54,016::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:21:54,016::task::993::Storage.TaskManager.Task::(_decref) Task=`583a4fe8-0944-4e27-b680-df272cf147ad`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:54,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:54,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116558 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:56,296::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,296::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) (252) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) (252) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) (252) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,297::__init__::225::IOProcess::(_processLogs) (252) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,298::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:21:56,298::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:56,299::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:21:56,305::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000200315 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:56,305::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,305::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:21:56,305::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,305::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,306::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:56,306::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:56,306::__init__::225::IOProcess::(_processLogs) (253) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:56,306::__init__::225::IOProcess::(_processLogs) (253) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:56,306::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(253) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,307::__init__::225::IOProcess::(_processLogs) (253) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,307::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:56,308::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:56,381::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,381::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:56,381::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,382::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,382::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:21:56,382::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:21:56,383::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,388::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:21:56,389::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000221335 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:21:56,389::__init__::225::IOProcess::(_processLogs) (254) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:56,390::__init__::225::IOProcess::(_processLogs) (254) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:56,390::__init__::225::IOProcess::(_processLogs) (254) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,390::__init__::225::IOProcess::(_processLogs) (254) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,390::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:56,390::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) (255) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:21:56,391::__init__::225::IOProcess::(_processLogs) (255) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:21:56,392::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(255) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,393::__init__::225::IOProcess::(_processLogs) (255) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,393::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:56,393::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:21:56,393::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,393::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:21:56,393::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) (256) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) (256) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) (256) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) (256) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:56,394::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:56,395::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,395::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:21:56,395::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,395::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) (257) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) (257) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) (257) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) (257) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:21:56,396::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:21:56,397::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:21:56,397::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:21:56,397::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:21:56,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) (258) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) (258) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) (258) Building response >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) (258) Queuing response >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:21:56,398::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:21:56,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:56,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119999 s, 85.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:21:58,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0d0846f-6f2b-470a-b7da-0c7004a283f5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:21:58,347::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:21:58,348::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000200315', 'lastCheck': '2.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221335', 'lastCheck': '2.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:58,348::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0d0846f-6f2b-470a-b7da-0c7004a283f5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000200315', 'lastCheck': '2.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221335', 'lastCheck': '2.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:21:58,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0d0846f-6f2b-470a-b7da-0c7004a283f5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:21:58,348::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:21:58,348::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:21:58,348::task::993::Storage.TaskManager.Task::(_decref) Task=`a0d0846f-6f2b-470a-b7da-0c7004a283f5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:21:58,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:21:58,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116739 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:00,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:00,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01256 s, 81.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:02,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:02,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185408 s, 55.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:04,074::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:22:04,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d5b5896-fb39-4b03-a25a-ba918248ada5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:04,074::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:22:04,075::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:04,075::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d5b5896-fb39-4b03-a25a-ba918248ada5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:04,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d5b5896-fb39-4b03-a25a-ba918248ada5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:04,075::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:04,075::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:04,075::task::993::Storage.TaskManager.Task::(_decref) Task=`4d5b5896-fb39-4b03-a25a-ba918248ada5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:22:04,122::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:22:04,122::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f2eedca-5585-4d0a-9f34-bbfcbea010b9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:22:04,123::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:22:04,123::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9f44a165-e080-418e-a7eb-401d3b12e050`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:22:04,123::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:22:04,123::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:22:04,123::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9f44a165-e080-418e-a7eb-401d3b12e050`::Granted request >Thread-46::DEBUG::2014-08-22 09:22:04,123::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5f2eedca-5585-4d0a-9f34-bbfcbea010b9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:22:04,123::task::993::Storage.TaskManager.Task::(_decref) Task=`5f2eedca-5585-4d0a-9f34-bbfcbea010b9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:22:04,125::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:04,125::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f2eedca-5585-4d0a-9f34-bbfcbea010b9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:04,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f2eedca-5585-4d0a-9f34-bbfcbea010b9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:22:04,126::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:22:04,126::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:22:04,126::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:22:04,126::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:22:04,126::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:22:04,126::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:22:04,126::task::993::Storage.TaskManager.Task::(_decref) Task=`5f2eedca-5585-4d0a-9f34-bbfcbea010b9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:04,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:04,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117531 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,308::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,309::__init__::225::IOProcess::(_processLogs) (259) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:06,309::__init__::225::IOProcess::(_processLogs) (259) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:06,309::__init__::225::IOProcess::(_processLogs) (259) Queuing response >Thread-21::DEBUG::2014-08-22 09:22:06,310::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:06,310::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:06,316::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:22:06,317::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000208695 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) (260) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:06,318::__init__::225::IOProcess::(_processLogs) (260) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:06,319::__init__::225::IOProcess::(_processLogs) (260) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:06,320::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:06,320::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:06,398::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:06,398::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:06,398::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,398::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,398::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,399::__init__::225::IOProcess::(_processLogs) (261) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:06,399::__init__::225::IOProcess::(_processLogs) (261) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:06,399::__init__::225::IOProcess::(_processLogs) (261) Building response >Thread-17::DEBUG::2014-08-22 09:22:06,399::__init__::225::IOProcess::(_processLogs) (261) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:06,399::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:22:06,400::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:22:06,406::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000252103 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:06,406::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:06,406::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:06,406::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,407::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,407::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,407::__init__::225::IOProcess::(_processLogs) (262) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:06,407::__init__::225::IOProcess::(_processLogs) (262) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:06,407::__init__::225::IOProcess::(_processLogs) (262) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:06,408::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:06,408::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:06,408::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:22:06,408::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) (263) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) (263) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) (263) Building response >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) (263) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:06,409::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:06,410::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:06,410::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:22:06,410::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:06,410::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) (264) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) (264) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) (264) Building response >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) (264) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:22:06,411::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:06,412::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:06,412::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:06,412::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:06,413::__init__::225::IOProcess::(_processLogs) (265) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:06,413::__init__::225::IOProcess::(_processLogs) (265) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:06,413::__init__::225::IOProcess::(_processLogs) (265) Building response >Thread-17::DEBUG::2014-08-22 09:22:06,413::__init__::225::IOProcess::(_processLogs) (265) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:06,413::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:06,413::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:22:07,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:07,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117989 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:09,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:09,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115141 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:11,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:11,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117662 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:13,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:13,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118538 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:13,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c5ae2be-bd68-4552-9f94-8c14241ea760`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:13,571::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:22:13,571::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208695', 'lastCheck': '7.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000252103', 'lastCheck': '7.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:13,571::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c5ae2be-bd68-4552-9f94-8c14241ea760`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208695', 'lastCheck': '7.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000252103', 'lastCheck': '7.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:13,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c5ae2be-bd68-4552-9f94-8c14241ea760`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:13,571::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:13,571::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:13,571::task::993::Storage.TaskManager.Task::(_decref) Task=`2c5ae2be-bd68-4552-9f94-8c14241ea760`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:22:14,176::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-47::DEBUG::2014-08-22 09:22:14,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`def21163-dedd-49dc-bfb1-251d92ec7c10`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:14,176::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:22:14,176::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:14,177::task::1191::Storage.TaskManager.Task::(prepare) Task=`def21163-dedd-49dc-bfb1-251d92ec7c10`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:14,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`def21163-dedd-49dc-bfb1-251d92ec7c10`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:14,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:14,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:14,177::task::993::Storage.TaskManager.Task::(_decref) Task=`def21163-dedd-49dc-bfb1-251d92ec7c10`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:22:14,212::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-46::DEBUG::2014-08-22 09:22:14,212::task::595::Storage.TaskManager.Task::(_updateState) Task=`20a7c85f-2161-4c59-a17c-44c5ed3934c5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:22:14,213::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:22:14,213::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9996cf68-1981-4c16-bab0-06aaecae574f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:22:14,213::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:22:14,213::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:22:14,213::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9996cf68-1981-4c16-bab0-06aaecae574f`::Granted request >Thread-46::DEBUG::2014-08-22 09:22:14,213::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`20a7c85f-2161-4c59-a17c-44c5ed3934c5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:22:14,213::task::993::Storage.TaskManager.Task::(_decref) Task=`20a7c85f-2161-4c59-a17c-44c5ed3934c5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:22:14,215::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:14,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`20a7c85f-2161-4c59-a17c-44c5ed3934c5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:14,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`20a7c85f-2161-4c59-a17c-44c5ed3934c5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:22:14,216::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:22:14,216::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:22:14,216::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:22:14,216::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:22:14,216::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:22:14,216::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:22:14,216::task::993::Storage.TaskManager.Task::(_decref) Task=`20a7c85f-2161-4c59-a17c-44c5ed3934c5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:15,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:15,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116028 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:16,320::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:16,320::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,320::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,320::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,321::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,321::__init__::225::IOProcess::(_processLogs) (266) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:16,321::__init__::225::IOProcess::(_processLogs) (266) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:16,321::__init__::225::IOProcess::(_processLogs) DEBUG|(266) Building response >Thread-21::DEBUG::2014-08-22 09:22:16,322::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:16,322::__init__::225::IOProcess::(_processLogs) (266) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:16,328::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:22:16,329::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000201333 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:16,329::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) (267) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:16,331::__init__::225::IOProcess::(_processLogs) (267) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:16,332::__init__::225::IOProcess::(_processLogs) DEBUG|(267) Building response >Thread-17::DEBUG::2014-08-22 09:22:16,333::__init__::225::IOProcess::(_processLogs) (267) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:16,333::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:16,333::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,413::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,414::__init__::225::IOProcess::(_processLogs) (268) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:16,414::__init__::225::IOProcess::(_processLogs) (268) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:16,414::__init__::225::IOProcess::(_processLogs) DEBUG|(268) Building response >Thread-17::DEBUG::2014-08-22 09:22:16,415::__init__::225::IOProcess::(_processLogs) (268) Queuing response >Thread-22::DEBUG::2014-08-22 09:22:16,415::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:16,415::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:16,420::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:22:16,421::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000219092 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:16,421::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:16,421::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) (269) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:16,422::__init__::225::IOProcess::(_processLogs) (269) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:16,423::__init__::225::IOProcess::(_processLogs) DEBUG|(269) Building response >Thread-17::DEBUG::2014-08-22 09:22:16,423::__init__::225::IOProcess::(_processLogs) (269) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:16,423::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:16,424::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:16,424::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:16,424::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:22:16,424::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,424::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) (270) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) (270) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) (270) Building response >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) (270) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:16,425::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:16,426::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:16,426::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:22:16,426::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,426::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) (271) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) (271) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) (271) Building response >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) (271) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:16,454::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:16,455::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:16,455::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:22:16,455::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:16,455::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) (272) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) (272) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) (272) Building response >Thread-17::DEBUG::2014-08-22 09:22:16,456::__init__::225::IOProcess::(_processLogs) (272) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:16,457::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:16,457::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:22:17,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:17,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119784 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:19,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:19,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121282 s, 84.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:21,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:21,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119491 s, 85.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:23,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:23,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116446 s, 87.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:24,298::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:22:24,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`a26c960d-826a-4b5c-8810-b0144d4f0d13`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:24,299::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:22:24,299::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:24,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`a26c960d-826a-4b5c-8810-b0144d4f0d13`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:24,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`a26c960d-826a-4b5c-8810-b0144d4f0d13`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:24,299::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:24,299::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:24,299::task::993::Storage.TaskManager.Task::(_decref) Task=`a26c960d-826a-4b5c-8810-b0144d4f0d13`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:22:24,335::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:22:24,335::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd49bb7c-963a-4e66-90fb-ccc4b2feca14`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:22:24,335::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:22:24,336::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`744b1ef0-5ee4-417c-8c88-2eb4e82f5c26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:22:24,336::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:22:24,336::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:22:24,336::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`744b1ef0-5ee4-417c-8c88-2eb4e82f5c26`::Granted request >Thread-46::DEBUG::2014-08-22 09:22:24,336::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cd49bb7c-963a-4e66-90fb-ccc4b2feca14`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:22:24,336::task::993::Storage.TaskManager.Task::(_decref) Task=`cd49bb7c-963a-4e66-90fb-ccc4b2feca14`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:22:24,338::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:24,338::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd49bb7c-963a-4e66-90fb-ccc4b2feca14`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:24,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd49bb7c-963a-4e66-90fb-ccc4b2feca14`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:22:24,339::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:22:24,339::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:22:24,339::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:22:24,339::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:22:24,339::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:22:24,339::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:22:24,339::task::993::Storage.TaskManager.Task::(_decref) Task=`cd49bb7c-963a-4e66-90fb-ccc4b2feca14`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:25,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:25,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116672 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:26,333::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,333::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:26,333::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) (273) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) (273) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:26,334::__init__::225::IOProcess::(_processLogs) (273) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,335::__init__::225::IOProcess::(_processLogs) (273) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,335::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:22:26,335::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:26,336::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:22:26,342::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189304 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) (274) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:26,343::__init__::225::IOProcess::(_processLogs) (274) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:26,344::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|(274) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,345::__init__::225::IOProcess::(_processLogs) (274) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,345::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:26,345::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:26,456::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,456::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:26,457::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,457::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,458::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:22:26,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:26,459::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,465::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:22:26,465::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000241991 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:26,466::__init__::225::IOProcess::(_processLogs) (275) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:26,466::__init__::225::IOProcess::(_processLogs) (275) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) (275) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) (275) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,467::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:26,468::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,468::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:26,468::__init__::225::IOProcess::(_processLogs) (276) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:26,468::__init__::225::IOProcess::(_processLogs) (276) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:26,468::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|(276) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,469::__init__::225::IOProcess::(_processLogs) (276) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,469::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:26,469::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:26,469::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,470::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:22:26,470::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,470::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,470::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:26,470::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) (277) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) (277) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) (277) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) (277) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:22:26,471::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,472::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,472::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:26,472::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,472::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:26,472::__init__::225::IOProcess::(_processLogs) (278) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) (278) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) (278) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) (278) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:22:26,473::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:26,474::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:26,474::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:26,474::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:26,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:26,474::__init__::225::IOProcess::(_processLogs) (279) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:26,474::__init__::225::IOProcess::(_processLogs) (279) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:26,475::__init__::225::IOProcess::(_processLogs) (279) Building response >Thread-17::DEBUG::2014-08-22 09:22:26,475::__init__::225::IOProcess::(_processLogs) (279) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:26,475::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:26,475::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:22:27,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:27,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162899 s, 62.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:28,795::task::595::Storage.TaskManager.Task::(_updateState) Task=`69181331-2aee-466b-8505-c7ecb993dc83`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:28,795::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:22:28,796::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189304', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000241991', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:28,796::task::1191::Storage.TaskManager.Task::(prepare) Task=`69181331-2aee-466b-8505-c7ecb993dc83`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189304', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000241991', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:28,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`69181331-2aee-466b-8505-c7ecb993dc83`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:28,796::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:28,796::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:28,796::task::993::Storage.TaskManager.Task::(_decref) Task=`69181331-2aee-466b-8505-c7ecb993dc83`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:29,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:29,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116059 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:31,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:31,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011998 s, 85.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:33,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:33,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137684 s, 74.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:34,620::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:22:34,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d3b0885-ab55-4a2d-be3a-2aa54bc5621f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:34,620::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:22:34,620::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:34,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d3b0885-ab55-4a2d-be3a-2aa54bc5621f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:34,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d3b0885-ab55-4a2d-be3a-2aa54bc5621f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:34,621::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:34,621::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:34,621::task::993::Storage.TaskManager.Task::(_decref) Task=`7d3b0885-ab55-4a2d-be3a-2aa54bc5621f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:22:34,655::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:22:34,655::task::595::Storage.TaskManager.Task::(_updateState) Task=`7160f550-fbe1-4b65-b253-19303bc7cce8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:22:34,655::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:22:34,656::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`25aad6f9-e6be-4dd9-ae3a-1b43a93d190f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:22:34,656::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:22:34,656::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:22:34,656::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`25aad6f9-e6be-4dd9-ae3a-1b43a93d190f`::Granted request >Thread-46::DEBUG::2014-08-22 09:22:34,656::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7160f550-fbe1-4b65-b253-19303bc7cce8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:22:34,656::task::993::Storage.TaskManager.Task::(_decref) Task=`7160f550-fbe1-4b65-b253-19303bc7cce8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:22:34,658::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:34,658::task::1191::Storage.TaskManager.Task::(prepare) Task=`7160f550-fbe1-4b65-b253-19303bc7cce8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:34,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`7160f550-fbe1-4b65-b253-19303bc7cce8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:22:34,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:22:34,659::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:22:34,659::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:22:34,659::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:22:34,659::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:22:34,659::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:22:34,659::task::993::Storage.TaskManager.Task::(_decref) Task=`7160f550-fbe1-4b65-b253-19303bc7cce8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:35,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:35,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116285 s, 88.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:22:36,345::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:22:36,347::__init__::225::IOProcess::(_processLogs) (273) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,347::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:22:36,347::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,347::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,348::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,349::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,349::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,350::__init__::225::IOProcess::(_processLogs) (280) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:22:36,351::__init__::225::IOProcess::(_processLogs) (280) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:22:36,351::__init__::225::IOProcess::(_processLogs) (280) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,352::__init__::225::IOProcess::(_processLogs) (280) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,353::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,353::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:22:36,354::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:22:36,355::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,356::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-21::DEBUG::2014-08-22 09:22:36,356::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:22:36,356::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,357::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,357::__init__::225::IOProcess::(_processLogs) (281) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) (281) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) (281) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) (281) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) Message size is 169 >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,358::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,359::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,359::__init__::225::IOProcess::(_processLogs) (282) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:22:36,359::__init__::225::IOProcess::(_processLogs) (282) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:22:36,377::__init__::225::IOProcess::(_processLogs) (282) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,377::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:22:36,378::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:22:36,378::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:36,378::__init__::225::IOProcess::(_processLogs) Message size is 193 >Thread-17::DEBUG::2014-08-22 09:22:36,379::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,379::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,379::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,379::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,379::__init__::225::IOProcess::(_processLogs) (283) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:22:36,379::__init__::225::IOProcess::(_processLogs) (283) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:22:36,380::__init__::225::IOProcess::(_processLogs) (283) Queuing response >Thread-21::DEBUG::2014-08-22 09:22:36,380::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:22:36,380::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,381::__init__::225::IOProcess::(_processLogs) Sending response sized 603 >Thread-17::DEBUG::2014-08-22 09:22:36,381::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:22:36,381::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,381::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) (284) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) (284) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) (284) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) (284) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,382::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) (285) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) (285) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 09:22:36,383::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:22:36,383::__init__::225::IOProcess::(_processLogs) (285) Queuing response >Thread-21::INFO::2014-08-22 09:22:36,384::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:22:36,384::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 09:22:36,384::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:22:36,385::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-17::DEBUG::2014-08-22 09:22:36,385::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:36,385::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,385::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,385::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,385::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,386::__init__::225::IOProcess::(_processLogs) (286) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:36,386::__init__::225::IOProcess::(_processLogs) (286) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:36,386::__init__::225::IOProcess::(_processLogs) (286) Queuing response >Thread-21::DEBUG::2014-08-22 09:22:36,387::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:36,387::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,393::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:22:36,393::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000326904 s, 1.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:36,394::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:36,394::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,394::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,395::__init__::225::IOProcess::(_processLogs) (287) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:36,395::__init__::225::IOProcess::(_processLogs) (287) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:36,395::__init__::225::IOProcess::(_processLogs) (287) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,396::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,396::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:22:36,474::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) (288) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) (288) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) (288) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,476::__init__::225::IOProcess::(_processLogs) (288) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,477::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,477::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,477::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:22:36,477::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) (289) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) (289) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:22:36,478::__init__::225::IOProcess::(_processLogs) (289) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,479::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,480::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-22::DEBUG::2014-08-22 09:22:36,481::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) Message size is 171 >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) (290) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:22:36,482::__init__::225::IOProcess::(_processLogs) (290) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:22:36,494::__init__::225::IOProcess::(_processLogs) (290) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,494::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,494::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Sending response sized 73 >Thread-22::DEBUG::2014-08-22 09:22:36,494::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:22:36,495::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,495::__init__::225::IOProcess::(_processLogs) Message size is 195 >Thread-17::DEBUG::2014-08-22 09:22:36,495::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,495::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,496::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,496::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,496::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,496::__init__::225::IOProcess::(_processLogs) (291) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:22:36,496::__init__::225::IOProcess::(_processLogs) (291) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:22:36,496::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|(291) Building response >Thread-22::DEBUG::2014-08-22 09:22:36,496::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:22:36,497::__init__::225::IOProcess::(_processLogs) (291) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,497::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,498::__init__::225::IOProcess::(_processLogs) Sending response sized 831 >Thread-17::DEBUG::2014-08-22 09:22:36,498::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,498::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-17::DEBUG::2014-08-22 09:22:36,498::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,498::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) (292) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) (292) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) (292) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) (292) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,499::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) (293) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:22:36,500::__init__::225::IOProcess::(_processLogs) (293) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:22:36,501::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|(293) Building response >Thread-22::DEBUG::2014-08-22 09:22:36,501::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:22:36,501::__init__::225::IOProcess::(_processLogs) (293) Queuing response >Thread-22::INFO::2014-08-22 09:22:36,501::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:22:36,502::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 09:22:36,502::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:22:36,502::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-17::DEBUG::2014-08-22 09:22:36,502::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,503::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:36,503::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,503::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,504::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:22:36,504::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:36,504::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,510::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:22:36,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000242929 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:36,511::__init__::225::IOProcess::(_processLogs) (294) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:36,512::__init__::225::IOProcess::(_processLogs) (294) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:36,512::__init__::225::IOProcess::(_processLogs) (294) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,512::__init__::225::IOProcess::(_processLogs) (294) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,512::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:36,512::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) (295) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:36,513::__init__::225::IOProcess::(_processLogs) (295) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:36,514::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|(295) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,514::__init__::225::IOProcess::(_processLogs) (295) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,514::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,514::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:36,515::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,515::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:22:36,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,515::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) (296) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) (296) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) (296) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) (296) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:36,516::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,517::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:22:36,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,517::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,517::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,517::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,517::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) (297) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) (297) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) (297) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) (297) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:22:36,518::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:36,519::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:36,519::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:36,519::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:36,519::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:36,520::__init__::225::IOProcess::(_processLogs) (298) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:36,520::__init__::225::IOProcess::(_processLogs) (298) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:36,520::__init__::225::IOProcess::(_processLogs) (298) Building response >Thread-17::DEBUG::2014-08-22 09:22:36,520::__init__::225::IOProcess::(_processLogs) (298) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:36,520::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:36,520::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:22:37,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:37,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118715 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:39,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:39,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011593 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:41,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:41,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117249 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:43,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:43,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117774 s, 86.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:44,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`5945f5dc-cedf-43ff-b545-4ebcda19f0f9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:44,056::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:22:44,056::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000326904', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000242929', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:44,056::task::1191::Storage.TaskManager.Task::(prepare) Task=`5945f5dc-cedf-43ff-b545-4ebcda19f0f9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000326904', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000242929', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:44,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`5945f5dc-cedf-43ff-b545-4ebcda19f0f9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:44,056::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:44,056::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:44,056::task::993::Storage.TaskManager.Task::(_decref) Task=`5945f5dc-cedf-43ff-b545-4ebcda19f0f9`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:22:44,728::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:22:44,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f678006-a3f2-4b70-b597-983ae5f16a51`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:44,728::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:22:44,728::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:44,728::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f678006-a3f2-4b70-b597-983ae5f16a51`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:44,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f678006-a3f2-4b70-b597-983ae5f16a51`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:44,728::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:44,729::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:44,729::task::993::Storage.TaskManager.Task::(_decref) Task=`2f678006-a3f2-4b70-b597-983ae5f16a51`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:22:44,750::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:22:44,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eec70a6-0927-4882-a19b-e1996795109d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:22:44,751::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:22:44,751::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`141e2b74-7f7e-459e-bdaf-5660d4008b9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:22:44,751::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:22:44,751::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:22:44,751::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`141e2b74-7f7e-459e-bdaf-5660d4008b9c`::Granted request >Thread-46::DEBUG::2014-08-22 09:22:44,752::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1eec70a6-0927-4882-a19b-e1996795109d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:22:44,752::task::993::Storage.TaskManager.Task::(_decref) Task=`1eec70a6-0927-4882-a19b-e1996795109d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:22:44,754::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:44,754::task::1191::Storage.TaskManager.Task::(prepare) Task=`1eec70a6-0927-4882-a19b-e1996795109d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:44,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eec70a6-0927-4882-a19b-e1996795109d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:22:44,754::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:22:44,754::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:22:44,754::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:22:44,754::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:22:44,754::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:22:44,754::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:22:44,754::task::993::Storage.TaskManager.Task::(_decref) Task=`1eec70a6-0927-4882-a19b-e1996795109d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:45,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:45,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115903 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) (299) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:46,397::__init__::225::IOProcess::(_processLogs) (299) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:46,398::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|(299) Building response >Thread-21::DEBUG::2014-08-22 09:22:46,399::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:46,399::__init__::225::IOProcess::(_processLogs) (299) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,405::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,406::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:22:46,406::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00018743 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:46,407::__init__::225::IOProcess::(_processLogs) (290) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,407::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:46,407::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,408::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:46,408::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,408::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:46,408::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,408::__init__::225::IOProcess::(_processLogs) (300) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:46,408::__init__::225::IOProcess::(_processLogs) (300) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:46,409::__init__::225::IOProcess::(_processLogs) DEBUG|(300) Building response >Thread-17::DEBUG::2014-08-22 09:22:46,409::__init__::225::IOProcess::(_processLogs) (300) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,409::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,410::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:46,520::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,520::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:46,520::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,521::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:22:46,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:46,521::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,527::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:22:46,528::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000203418 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:46,529::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) (301) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) (301) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) (301) Building response >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) (301) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:46,530::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,531::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:46,531::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,531::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:46,531::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,531::__init__::225::IOProcess::(_processLogs) (302) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:46,531::__init__::225::IOProcess::(_processLogs) (302) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:46,532::__init__::225::IOProcess::(_processLogs) DEBUG|(302) Building response >Thread-17::DEBUG::2014-08-22 09:22:46,532::__init__::225::IOProcess::(_processLogs) (302) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,532::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,532::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:46,533::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,533::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:22:46,533::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,533::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:46,533::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,533::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) (303) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) (303) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) (303) Building response >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) (303) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:22:46,534::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) (304) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) (304) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:46,535::__init__::225::IOProcess::(_processLogs) (304) Building response >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) (304) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:46,536::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) (305) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) (305) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) (305) Building response >Thread-17::DEBUG::2014-08-22 09:22:46,537::__init__::225::IOProcess::(_processLogs) (305) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:46,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:46,538::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:22:48,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:48,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120287 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:50,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:50,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115539 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:52,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:52,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118113 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:22:54,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:54,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116107 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:54,804::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:22:54,805::task::595::Storage.TaskManager.Task::(_updateState) Task=`53adb5cd-87fd-4e91-b1ed-0c124f4145d0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:54,805::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:22:54,805::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:54,805::task::1191::Storage.TaskManager.Task::(prepare) Task=`53adb5cd-87fd-4e91-b1ed-0c124f4145d0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:22:54,805::task::595::Storage.TaskManager.Task::(_updateState) Task=`53adb5cd-87fd-4e91-b1ed-0c124f4145d0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:54,805::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:54,805::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:54,805::task::993::Storage.TaskManager.Task::(_decref) Task=`53adb5cd-87fd-4e91-b1ed-0c124f4145d0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:22:54,829::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:22:54,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2a65902-752d-43a0-8954-ebc5edd1b8fd`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:22:54,830::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:22:54,830::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d5f97e8f-9822-4bb0-bd6e-0ea2c4827a37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:22:54,830::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:22:54,830::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:22:54,830::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d5f97e8f-9822-4bb0-bd6e-0ea2c4827a37`::Granted request >Thread-46::DEBUG::2014-08-22 09:22:54,831::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f2a65902-752d-43a0-8954-ebc5edd1b8fd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:22:54,831::task::993::Storage.TaskManager.Task::(_decref) Task=`f2a65902-752d-43a0-8954-ebc5edd1b8fd`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:22:54,833::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:54,833::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2a65902-752d-43a0-8954-ebc5edd1b8fd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:22:54,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2a65902-752d-43a0-8954-ebc5edd1b8fd`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:22:54,833::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:22:54,833::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:22:54,833::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:22:54,833::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:22:54,833::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:22:54,833::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:22:54,834::task::993::Storage.TaskManager.Task::(_decref) Task=`f2a65902-752d-43a0-8954-ebc5edd1b8fd`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:22:56,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:56,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117143 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,410::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:56,411::__init__::225::IOProcess::(_processLogs) (306) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:56,411::__init__::225::IOProcess::(_processLogs) (306) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:56,411::__init__::225::IOProcess::(_processLogs) DEBUG|(306) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,412::__init__::225::IOProcess::(_processLogs) (306) Queuing response >Thread-21::DEBUG::2014-08-22 09:22:56,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:56,413::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,419::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:22:56,420::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000201213 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:56,420::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,420::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) (307) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:56,421::__init__::225::IOProcess::(_processLogs) (307) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:56,422::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(307) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,423::__init__::225::IOProcess::(_processLogs) (307) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:56,423::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,423::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:56,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,538::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:56,538::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,538::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,539::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:22:56,539::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:22:56,539::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,545::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:22:56,545::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216517 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:22:56,546::__init__::225::IOProcess::(_processLogs) (308) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) (308) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) (308) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) (308) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,547::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) (309) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:22:56,548::__init__::225::IOProcess::(_processLogs) (309) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:22:56,549::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(309) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,549::__init__::225::IOProcess::(_processLogs) (309) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:56,549::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,549::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:22:56,550::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,550::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:22:56,550::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,550::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) (310) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) (310) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) (310) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) (310) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,551::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:22:56,552::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,552::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,552::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:56,552::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,552::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) (311) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) (311) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) (311) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) (311) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:22:56,553::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) (312) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) (312) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) (312) Building response >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) (312) Queuing response >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:22:56,554::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:22:58,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:22:58,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117858 s, 86.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:22:59,258::task::595::Storage.TaskManager.Task::(_updateState) Task=`01a123ac-88ac-4d6c-9bb6-ab97c1be6030`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:22:59,258::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:22:59,258::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000201213', 'lastCheck': '2.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216517', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:59,258::task::1191::Storage.TaskManager.Task::(prepare) Task=`01a123ac-88ac-4d6c-9bb6-ab97c1be6030`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000201213', 'lastCheck': '2.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216517', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:22:59,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`01a123ac-88ac-4d6c-9bb6-ab97c1be6030`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:22:59,259::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:22:59,259::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:22:59,259::task::993::Storage.TaskManager.Task::(_decref) Task=`01a123ac-88ac-4d6c-9bb6-ab97c1be6030`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:23:00,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:00,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011594 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:02,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:02,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211539 s, 48.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:04,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:04,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118543 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:04,907::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:23:04,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`094d35f9-c44d-44fa-ba69-f0f6511b4c97`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:04,908::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:23:04,908::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:04,908::task::1191::Storage.TaskManager.Task::(prepare) Task=`094d35f9-c44d-44fa-ba69-f0f6511b4c97`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:04,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`094d35f9-c44d-44fa-ba69-f0f6511b4c97`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:04,908::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:04,908::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:04,908::task::993::Storage.TaskManager.Task::(_decref) Task=`094d35f9-c44d-44fa-ba69-f0f6511b4c97`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:23:04,942::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:23:04,942::task::595::Storage.TaskManager.Task::(_updateState) Task=`96062a1d-ca6f-4307-9f07-6d58cdd5cde2`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:23:04,942::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:23:04,943::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f03df1bb-3b73-49fb-a595-e0b62205c2d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:23:04,943::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:23:04,943::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:23:04,943::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f03df1bb-3b73-49fb-a595-e0b62205c2d4`::Granted request >Thread-46::DEBUG::2014-08-22 09:23:04,943::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`96062a1d-ca6f-4307-9f07-6d58cdd5cde2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:23:04,943::task::993::Storage.TaskManager.Task::(_decref) Task=`96062a1d-ca6f-4307-9f07-6d58cdd5cde2`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:23:04,945::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:04,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`96062a1d-ca6f-4307-9f07-6d58cdd5cde2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:04,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`96062a1d-ca6f-4307-9f07-6d58cdd5cde2`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:23:04,946::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:23:04,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:23:04,946::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:23:04,946::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:23:04,946::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:23:04,946::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:23:04,946::task::993::Storage.TaskManager.Task::(_decref) Task=`96062a1d-ca6f-4307-9f07-6d58cdd5cde2`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:23:06,423::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:06,423::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:06,423::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:06,423::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,424::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,424::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:06,424::__init__::225::IOProcess::(_processLogs) (313) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:06,424::__init__::225::IOProcess::(_processLogs) (313) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:06,424::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(313) Building response >Thread-21::DEBUG::2014-08-22 09:23:06,425::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:06,425::__init__::225::IOProcess::(_processLogs) (313) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,431::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:23:06,432::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000221673 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:06,432::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) (314) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) (314) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:06,434::__init__::225::IOProcess::(_processLogs) (314) Building response >Thread-17::DEBUG::2014-08-22 09:23:06,435::__init__::225::IOProcess::(_processLogs) (314) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,435::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:06,435::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:23:06,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:06,578::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:06,578::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:06,578::__init__::225::IOProcess::(_processLogs) Marshaling message... >Dummy-45::DEBUG::2014-08-22 09:23:06,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011695 s, 87.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:06,579::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:23:06,580::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:06,580::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,586::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,588::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:23:06,588::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000315225 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:06,589::__init__::225::IOProcess::(_processLogs) (315) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:06,590::__init__::225::IOProcess::(_processLogs) (315) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:06,590::__init__::225::IOProcess::(_processLogs) (315) Building response >Thread-17::DEBUG::2014-08-22 09:23:06,591::__init__::225::IOProcess::(_processLogs) (315) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,591::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:06,592::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:06,592::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:06,593::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:06,593::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:06,595::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:06,596::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,596::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,597::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:06,597::__init__::225::IOProcess::(_processLogs) (316) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:06,598::__init__::225::IOProcess::(_processLogs) (316) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:06,598::__init__::225::IOProcess::(_processLogs) (316) Building response >Thread-17::DEBUG::2014-08-22 09:23:06,598::__init__::225::IOProcess::(_processLogs) (316) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,599::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:06,599::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:06,600::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:06,600::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:23:06,601::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:06,602::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:06,602::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,603::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,603::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:06,603::__init__::225::IOProcess::(_processLogs) (317) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:06,604::__init__::225::IOProcess::(_processLogs) (317) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:06,604::__init__::225::IOProcess::(_processLogs) (317) Building response >Thread-17::DEBUG::2014-08-22 09:23:06,605::__init__::225::IOProcess::(_processLogs) (317) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,605::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:06,605::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:06,606::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:06,606::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:23:06,607::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:06,608::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:06,608::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,609::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,609::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:06,609::__init__::225::IOProcess::(_processLogs) (318) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:06,610::__init__::225::IOProcess::(_processLogs) (318) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:06,610::__init__::225::IOProcess::(_processLogs) (318) Building response >Thread-17::DEBUG::2014-08-22 09:23:06,611::__init__::225::IOProcess::(_processLogs) (318) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,611::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:06,611::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:06,612::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:06,612::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:23:06,613::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:06,614::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:06,615::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:06,615::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:06,615::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:06,616::__init__::225::IOProcess::(_processLogs) (319) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:06,616::__init__::225::IOProcess::(_processLogs) (319) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:06,617::__init__::225::IOProcess::(_processLogs) (319) Building response >Thread-17::DEBUG::2014-08-22 09:23:06,617::__init__::225::IOProcess::(_processLogs) (319) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:06,617::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:06,618::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:23:08,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:08,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118642 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:10,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:10,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116767 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:12,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:12,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159044 s, 64.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:14,468::task::595::Storage.TaskManager.Task::(_updateState) Task=`c31a7fec-7126-4d99-a0e2-6367b8d17a76`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:14,468::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:23:14,469::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000221673', 'lastCheck': '8.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000315225', 'lastCheck': '7.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:23:14,469::task::1191::Storage.TaskManager.Task::(prepare) Task=`c31a7fec-7126-4d99-a0e2-6367b8d17a76`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000221673', 'lastCheck': '8.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000315225', 'lastCheck': '7.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:23:14,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`c31a7fec-7126-4d99-a0e2-6367b8d17a76`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:14,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:14,469::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:14,469::task::993::Storage.TaskManager.Task::(_decref) Task=`c31a7fec-7126-4d99-a0e2-6367b8d17a76`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:23:14,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:14,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114042 s, 89.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:15,042::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:23:15,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`62be068d-7429-46a4-89bc-7acd550f5c73`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:15,043::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:23:15,043::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:15,043::task::1191::Storage.TaskManager.Task::(prepare) Task=`62be068d-7429-46a4-89bc-7acd550f5c73`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:15,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`62be068d-7429-46a4-89bc-7acd550f5c73`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:15,043::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:15,044::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:15,044::task::993::Storage.TaskManager.Task::(_decref) Task=`62be068d-7429-46a4-89bc-7acd550f5c73`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:23:15,074::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:23:15,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`1274e3ab-0bb5-4dab-877c-5456295a6195`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:23:15,075::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:23:15,075::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2ab0cc53-29ef-40d5-a5a9-8057eb51fbb5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:23:15,075::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:23:15,075::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:23:15,075::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2ab0cc53-29ef-40d5-a5a9-8057eb51fbb5`::Granted request >Thread-46::DEBUG::2014-08-22 09:23:15,075::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1274e3ab-0bb5-4dab-877c-5456295a6195`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:23:15,076::task::993::Storage.TaskManager.Task::(_decref) Task=`1274e3ab-0bb5-4dab-877c-5456295a6195`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:23:15,077::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:15,078::task::1191::Storage.TaskManager.Task::(prepare) Task=`1274e3ab-0bb5-4dab-877c-5456295a6195`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:15,078::task::595::Storage.TaskManager.Task::(_updateState) Task=`1274e3ab-0bb5-4dab-877c-5456295a6195`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:23:15,078::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:23:15,078::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:23:15,078::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:23:15,078::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:23:15,078::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:23:15,078::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:23:15,078::task::993::Storage.TaskManager.Task::(_decref) Task=`1274e3ab-0bb5-4dab-877c-5456295a6195`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:23:16,436::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,436::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) (320) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:16,437::__init__::225::IOProcess::(_processLogs) (320) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:16,438::__init__::225::IOProcess::(_processLogs) DEBUG|(320) Building response >Thread-21::DEBUG::2014-08-22 09:23:16,438::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:16,439::__init__::225::IOProcess::(_processLogs) (320) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,445::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,446::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:23:16,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000200204 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,447::__init__::225::IOProcess::(_processLogs) (321) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:16,448::__init__::225::IOProcess::(_processLogs) (321) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:16,448::__init__::225::IOProcess::(_processLogs) (321) Building response >Thread-17::DEBUG::2014-08-22 09:23:16,448::__init__::225::IOProcess::(_processLogs) (321) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,448::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,448::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:16,615::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,615::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:16,615::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,615::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,615::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,615::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,616::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,616::__init__::225::IOProcess::(_processLogs) (322) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:16,616::__init__::225::IOProcess::(_processLogs) (322) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:16,616::__init__::225::IOProcess::(_processLogs) DEBUG|(322) Building response >Thread-22::DEBUG::2014-08-22 09:23:16,617::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:16,617::__init__::225::IOProcess::(_processLogs) (322) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,623::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,623::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:23:16,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000205524 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,624::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,625::__init__::225::IOProcess::(_processLogs) (323) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:16,625::__init__::225::IOProcess::(_processLogs) (323) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:16,625::__init__::225::IOProcess::(_processLogs) DEBUG|(323) Building response >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) (323) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,626::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) (324) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) (324) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) (324) Building response >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) (324) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,627::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:16,628::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,628::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:23:16,628::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,628::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) (325) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) (325) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) (325) Building response >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) (325) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,629::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:16,630::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:16,630::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:23:16,630::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:16,630::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:16,630::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) (326) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) (326) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) (326) Building response >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) (326) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:16,631::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:16,632::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:23:16,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:16,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115459 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:18,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:18,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011638 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:20,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:20,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116254 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:22,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:23,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117234 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:25,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:25,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118318 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:25,127::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:23:25,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`63617cab-86ae-4e54-bd1b-d96d31fc6f9e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:25,127::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:23:25,128::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:25,128::task::1191::Storage.TaskManager.Task::(prepare) Task=`63617cab-86ae-4e54-bd1b-d96d31fc6f9e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:25,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`63617cab-86ae-4e54-bd1b-d96d31fc6f9e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:25,128::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:25,128::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:25,128::task::993::Storage.TaskManager.Task::(_decref) Task=`63617cab-86ae-4e54-bd1b-d96d31fc6f9e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:23:25,150::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:23:25,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`551e11c8-1fd3-458a-aae9-edbd27fe154a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:23:25,150::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:23:25,150::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a771fcb5-9cfa-4be8-94b1-70c27d1ad590`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:23:25,150::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:23:25,150::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:23:25,151::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a771fcb5-9cfa-4be8-94b1-70c27d1ad590`::Granted request >Thread-46::DEBUG::2014-08-22 09:23:25,151::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`551e11c8-1fd3-458a-aae9-edbd27fe154a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:23:25,151::task::993::Storage.TaskManager.Task::(_decref) Task=`551e11c8-1fd3-458a-aae9-edbd27fe154a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:23:25,153::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:25,153::task::1191::Storage.TaskManager.Task::(prepare) Task=`551e11c8-1fd3-458a-aae9-edbd27fe154a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:25,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`551e11c8-1fd3-458a-aae9-edbd27fe154a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:23:25,153::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:23:25,153::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:23:25,153::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:23:25,154::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:23:25,154::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:23:25,154::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:23:25,154::task::993::Storage.TaskManager.Task::(_decref) Task=`551e11c8-1fd3-458a-aae9-edbd27fe154a`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) (327) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:26,450::__init__::225::IOProcess::(_processLogs) (327) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:26,451::__init__::225::IOProcess::(_processLogs) DEBUG|(327) Building response >Thread-21::DEBUG::2014-08-22 09:23:26,452::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:26,452::__init__::225::IOProcess::(_processLogs) (327) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,458::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,459::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:23:26,459::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196571 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:26,460::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,460::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:26,460::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,460::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,461::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,461::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,461::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,461::__init__::225::IOProcess::(_processLogs) (328) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:26,461::__init__::225::IOProcess::(_processLogs) (328) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:26,462::__init__::225::IOProcess::(_processLogs) DEBUG|(328) Building response >Thread-17::DEBUG::2014-08-22 09:23:26,462::__init__::225::IOProcess::(_processLogs) (328) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,462::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,462::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:26,631::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,631::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:26,631::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) (329) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) (329) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) (329) Building response >Thread-17::DEBUG::2014-08-22 09:23:26,632::__init__::225::IOProcess::(_processLogs) (329) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,633::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,633::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:23:26,633::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:23:26,641::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000246512 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:26,642::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,643::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:26,643::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,643::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,643::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,644::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,644::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,644::__init__::225::IOProcess::(_processLogs) (330) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:26,644::__init__::225::IOProcess::(_processLogs) (330) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) (330) Building response >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) (330) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:23:26,645::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) (331) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) (331) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) (331) Building response >Thread-17::DEBUG::2014-08-22 09:23:26,646::__init__::225::IOProcess::(_processLogs) (331) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,647::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,647::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:26,647::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,647::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:23:26,647::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,647::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) (332) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) (332) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) (332) Building response >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) (332) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,648::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:26,649::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:26,649::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:23:26,649::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:26,649::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:26,649::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) (333) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) (333) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) (333) Building response >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) (333) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:26,650::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:23:27,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:27,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119213 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:29,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:29,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116798 s, 87.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:29,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e15b12e-0c3d-4e3c-90d3-80af6b819ea6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:29,654::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:23:29,655::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196571', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000246512', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:23:29,655::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e15b12e-0c3d-4e3c-90d3-80af6b819ea6`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196571', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000246512', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:23:29,655::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e15b12e-0c3d-4e3c-90d3-80af6b819ea6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:29,655::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:29,655::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:29,655::task::993::Storage.TaskManager.Task::(_decref) Task=`0e15b12e-0c3d-4e3c-90d3-80af6b819ea6`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:23:31,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:31,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118439 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:33,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:33,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119526 s, 85.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:35,203::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:23:35,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`99051e58-3387-4a16-988b-55a1ff1d5fec`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:35,203::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:23:35,203::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:35,203::task::1191::Storage.TaskManager.Task::(prepare) Task=`99051e58-3387-4a16-988b-55a1ff1d5fec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:35,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`99051e58-3387-4a16-988b-55a1ff1d5fec`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:35,204::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:35,204::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:35,204::task::993::Storage.TaskManager.Task::(_decref) Task=`99051e58-3387-4a16-988b-55a1ff1d5fec`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:23:35,266::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:23:35,267::task::595::Storage.TaskManager.Task::(_updateState) Task=`823f56e6-ad0b-48e1-a770-16fcde75134b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:23:35,267::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:23:35,267::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2775b18b-e494-4f74-aab0-0a65af50347c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:23:35,267::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:23:35,267::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:23:35,267::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2775b18b-e494-4f74-aab0-0a65af50347c`::Granted request >Thread-46::DEBUG::2014-08-22 09:23:35,268::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`823f56e6-ad0b-48e1-a770-16fcde75134b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:23:35,268::task::993::Storage.TaskManager.Task::(_decref) Task=`823f56e6-ad0b-48e1-a770-16fcde75134b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:23:35,270::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:35,270::task::1191::Storage.TaskManager.Task::(prepare) Task=`823f56e6-ad0b-48e1-a770-16fcde75134b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:35,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`823f56e6-ad0b-48e1-a770-16fcde75134b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:23:35,270::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:23:35,270::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:23:35,270::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:23:35,270::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:23:35,270::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:23:35,270::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:23:35,270::task::993::Storage.TaskManager.Task::(_decref) Task=`823f56e6-ad0b-48e1-a770-16fcde75134b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:23:35,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:35,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117663 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:36,463::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,463::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:36,463::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,463::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,463::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:36,463::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,464::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:36,464::__init__::225::IOProcess::(_processLogs) (334) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:36,464::__init__::225::IOProcess::(_processLogs) (334) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:36,464::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(334) Building response >Thread-21::DEBUG::2014-08-22 09:23:36,465::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:36,465::__init__::225::IOProcess::(_processLogs) (334) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,471::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:23:36,472::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00019985 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:36,472::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:36,473::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) (335) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:36,474::__init__::225::IOProcess::(_processLogs) (335) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:36,475::__init__::225::IOProcess::(_processLogs) DEBUG|(335) Building response >Thread-17::DEBUG::2014-08-22 09:23:36,476::__init__::225::IOProcess::(_processLogs) (335) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,476::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:36,476::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:36,650::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,650::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:36,650::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,651::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,651::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:23:36,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:36,652::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,658::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:23:36,658::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021351 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:36,659::__init__::225::IOProcess::(_processLogs) (336) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) (336) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) (336) Building response >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) (336) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,660::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:36,661::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) (337) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) (337) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) (337) Building response >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) (337) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:36,662::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:36,663::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,663::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:23:36,663::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,663::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,663::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:36,663::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) (338) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) (338) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) (338) Building response >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) (338) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:23:36,664::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,665::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,665::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:36,665::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,665::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:36,665::__init__::225::IOProcess::(_processLogs) (339) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:36,665::__init__::225::IOProcess::(_processLogs) (339) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) (339) Building response >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) (339) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:23:36,666::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) (340) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) (340) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:36,667::__init__::225::IOProcess::(_processLogs) (340) Building response >Thread-17::DEBUG::2014-08-22 09:23:36,668::__init__::225::IOProcess::(_processLogs) (340) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:36,668::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:36,668::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:23:37,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:37,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118567 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:39,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:39,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117046 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:41,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:41,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196304 s, 52.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:43,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:43,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119379 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:44,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`448eaa9e-4475-4319-b9f7-c6333c426bce`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:44,857::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:23:44,857::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00019985', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021351', 'lastCheck': '8.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:23:44,857::task::1191::Storage.TaskManager.Task::(prepare) Task=`448eaa9e-4475-4319-b9f7-c6333c426bce`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00019985', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021351', 'lastCheck': '8.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:23:44,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`448eaa9e-4475-4319-b9f7-c6333c426bce`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:44,857::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:44,857::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:44,857::task::993::Storage.TaskManager.Task::(_decref) Task=`448eaa9e-4475-4319-b9f7-c6333c426bce`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:23:45,364::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:23:45,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`98d678b8-0c38-473a-8ec4-a2ccea3b221f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:45,364::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:23:45,365::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:45,365::task::1191::Storage.TaskManager.Task::(prepare) Task=`98d678b8-0c38-473a-8ec4-a2ccea3b221f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:45,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`98d678b8-0c38-473a-8ec4-a2ccea3b221f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:45,365::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:45,365::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:45,365::task::993::Storage.TaskManager.Task::(_decref) Task=`98d678b8-0c38-473a-8ec4-a2ccea3b221f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:23:45,390::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:23:45,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`616065e2-6c21-4a6f-a9a7-b992920368ae`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:23:45,391::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:23:45,391::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7bae43be-75e4-4f84-9185-78c4d46d6de9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:23:45,391::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:23:45,391::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:23:45,391::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7bae43be-75e4-4f84-9185-78c4d46d6de9`::Granted request >Thread-46::DEBUG::2014-08-22 09:23:45,391::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`616065e2-6c21-4a6f-a9a7-b992920368ae`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:23:45,392::task::993::Storage.TaskManager.Task::(_decref) Task=`616065e2-6c21-4a6f-a9a7-b992920368ae`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:23:45,393::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:45,394::task::1191::Storage.TaskManager.Task::(prepare) Task=`616065e2-6c21-4a6f-a9a7-b992920368ae`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:45,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`616065e2-6c21-4a6f-a9a7-b992920368ae`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:23:45,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:23:45,394::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:23:45,394::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:23:45,394::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:23:45,394::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:23:45,394::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:23:45,394::task::993::Storage.TaskManager.Task::(_decref) Task=`616065e2-6c21-4a6f-a9a7-b992920368ae`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:23:45,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:45,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118224 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:46,476::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:46,476::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:46,476::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,476::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,476::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:46,477::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,477::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:46,477::__init__::225::IOProcess::(_processLogs) (341) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:46,477::__init__::225::IOProcess::(_processLogs) (341) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:46,477::__init__::225::IOProcess::(_processLogs) (341) Queuing response >Thread-21::DEBUG::2014-08-22 09:23:46,478::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:46,478::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,485::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:23:46,485::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000212576 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:46,486::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:46,486::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,486::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,486::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:46,486::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,486::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:46,487::__init__::225::IOProcess::(_processLogs) (342) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:46,487::__init__::225::IOProcess::(_processLogs) (342) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:46,487::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(342) Building response >Thread-17::DEBUG::2014-08-22 09:23:46,488::__init__::225::IOProcess::(_processLogs) (342) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:46,488::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,488::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:46,668::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:46,668::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:46,668::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,668::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,669::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:23:46,669::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:46,669::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,675::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:23:46,676::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214271 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:46,676::__init__::225::IOProcess::(_processLogs) (343) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:46,677::__init__::225::IOProcess::(_processLogs) (343) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:46,677::__init__::225::IOProcess::(_processLogs) (343) Building response >Thread-17::DEBUG::2014-08-22 09:23:46,677::__init__::225::IOProcess::(_processLogs) (343) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:46,677::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,677::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) (344) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:46,678::__init__::225::IOProcess::(_processLogs) (344) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:46,679::__init__::225::IOProcess::(_processLogs) DEBUG|(344) Building response >Thread-17::DEBUG::2014-08-22 09:23:46,679::__init__::225::IOProcess::(_processLogs) (344) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:46,680::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,680::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:46,680::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:46,680::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:23:46,680::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,680::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) (345) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) (345) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) (345) Building response >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) (345) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,681::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:46,682::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:46,682::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:23:46,682::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,682::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,682::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) (346) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) (346) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) (346) Building response >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) (346) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:46,683::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:23:46,684::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:46,684::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:46,684::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) (347) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) (347) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) (347) Building response >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) (347) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:46,685::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:23:47,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:47,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012047 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:49,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:49,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115777 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:51,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:51,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116372 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:53,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:53,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01181 s, 86.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:23:55,442::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:23:55,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1dbd9bd-b8b7-4bc4-ab81-bb95a17bce79`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:23:55,443::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:23:55,443::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:55,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`f1dbd9bd-b8b7-4bc4-ab81-bb95a17bce79`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:23:55,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1dbd9bd-b8b7-4bc4-ab81-bb95a17bce79`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:23:55,444::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:23:55,444::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:23:55,444::task::993::Storage.TaskManager.Task::(_decref) Task=`f1dbd9bd-b8b7-4bc4-ab81-bb95a17bce79`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:23:55,478::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:23:55,478::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f4e7f06-27d4-41bd-9b0e-7bab2a955072`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:23:55,478::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:23:55,478::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`16a73d39-1cda-40fb-a1e2-f326d979c33d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:23:55,478::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:23:55,478::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:23:55,479::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`16a73d39-1cda-40fb-a1e2-f326d979c33d`::Granted request >Thread-46::DEBUG::2014-08-22 09:23:55,479::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4f4e7f06-27d4-41bd-9b0e-7bab2a955072`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:23:55,479::task::993::Storage.TaskManager.Task::(_decref) Task=`4f4e7f06-27d4-41bd-9b0e-7bab2a955072`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:23:55,481::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:55,481::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f4e7f06-27d4-41bd-9b0e-7bab2a955072`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:23:55,481::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f4e7f06-27d4-41bd-9b0e-7bab2a955072`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:23:55,481::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:23:55,481::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:23:55,481::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:23:55,481::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:23:55,481::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:23:55,482::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:23:55,482::task::993::Storage.TaskManager.Task::(_decref) Task=`4f4e7f06-27d4-41bd-9b0e-7bab2a955072`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:23:55,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:55,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01165 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) (348) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:56,489::__init__::225::IOProcess::(_processLogs) (348) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:56,490::__init__::225::IOProcess::(_processLogs) (348) Queuing response >Thread-21::DEBUG::2014-08-22 09:23:56,491::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:23:56,491::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:56,497::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:23:56,498::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000154243 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) (349) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:56,499::__init__::225::IOProcess::(_processLogs) (349) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:56,500::__init__::225::IOProcess::(_processLogs) (349) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:56,501::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:56,501::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) (350) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) (350) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:56,685::__init__::225::IOProcess::(_processLogs) (350) Building response >Thread-17::DEBUG::2014-08-22 09:23:56,686::__init__::225::IOProcess::(_processLogs) (350) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:56,686::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:23:56,686::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:23:56,692::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218214 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:23:56,693::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:23:56,693::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:56,693::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) (351) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) (351) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) (351) Building response >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) (351) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:56,694::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:56,695::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:23:56,695::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:23:56,695::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:56,695::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) (352) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) (352) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) (352) Building response >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) (352) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:56,696::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,697::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,697::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,697::__init__::225::IOProcess::(_processLogs) (353) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:56,697::__init__::225::IOProcess::(_processLogs) (353) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:56,697::__init__::225::IOProcess::(_processLogs) (353) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:56,697::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) (354) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) (354) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) (354) Building response >Thread-17::DEBUG::2014-08-22 09:23:56,698::__init__::225::IOProcess::(_processLogs) (354) Queuing response >Thread-17::DEBUG::2014-08-22 09:23:56,699::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:23:57,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:57,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011852 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:23:59,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:23:59,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115824 s, 88.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:00,027::task::595::Storage.TaskManager.Task::(_updateState) Task=`73596ced-f984-4763-8aec-c999db948e53`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:00,027::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:24:00,027::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000154243', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218214', 'lastCheck': '3.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:00,027::task::1191::Storage.TaskManager.Task::(prepare) Task=`73596ced-f984-4763-8aec-c999db948e53`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000154243', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218214', 'lastCheck': '3.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:00,027::task::595::Storage.TaskManager.Task::(_updateState) Task=`73596ced-f984-4763-8aec-c999db948e53`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:00,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:00,028::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:00,028::task::993::Storage.TaskManager.Task::(_decref) Task=`73596ced-f984-4763-8aec-c999db948e53`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:24:01,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:01,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183103 s, 55.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:04,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:04,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117016 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:05,532::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:24:05,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`28766414-4cbb-40cb-8ab1-621cadac458e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:05,532::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:24:05,532::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:05,533::task::1191::Storage.TaskManager.Task::(prepare) Task=`28766414-4cbb-40cb-8ab1-621cadac458e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:05,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`28766414-4cbb-40cb-8ab1-621cadac458e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:05,533::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:05,533::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:05,533::task::993::Storage.TaskManager.Task::(_decref) Task=`28766414-4cbb-40cb-8ab1-621cadac458e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:24:05,565::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:24:05,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`d03a3e71-c9de-4cee-8510-47c870746665`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:24:05,565::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:24:05,566::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`22c000a0-16ca-4759-bfd0-9b51713fe241`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:24:05,566::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:24:05,566::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:24:05,566::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`22c000a0-16ca-4759-bfd0-9b51713fe241`::Granted request >Thread-46::DEBUG::2014-08-22 09:24:05,566::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d03a3e71-c9de-4cee-8510-47c870746665`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:24:05,566::task::993::Storage.TaskManager.Task::(_decref) Task=`d03a3e71-c9de-4cee-8510-47c870746665`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:24:05,568::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:05,568::task::1191::Storage.TaskManager.Task::(prepare) Task=`d03a3e71-c9de-4cee-8510-47c870746665`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:05,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`d03a3e71-c9de-4cee-8510-47c870746665`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:24:05,568::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:24:05,569::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:24:05,569::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:24:05,569::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:24:05,569::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:24:05,569::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:24:05,569::task::993::Storage.TaskManager.Task::(_decref) Task=`d03a3e71-c9de-4cee-8510-47c870746665`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:24:06,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:06,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011827 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:06,501::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:06,501::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:06,501::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:06,501::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,502::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,502::__init__::225::IOProcess::(_processLogs) (355) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:06,502::__init__::225::IOProcess::(_processLogs) (355) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:06,502::__init__::225::IOProcess::(_processLogs) (355) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,503::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:24:06,503::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:06,504::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:24:06,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000205248 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:06,510::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:06,510::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:06,511::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,511::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,511::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,511::__init__::225::IOProcess::(_processLogs) (356) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:06,511::__init__::225::IOProcess::(_processLogs) (356) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:06,512::__init__::225::IOProcess::(_processLogs) (356) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:06,512::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:06,699::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:06,700::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:06,700::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,700::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,700::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,700::__init__::225::IOProcess::(_processLogs) (357) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:06,700::__init__::225::IOProcess::(_processLogs) (357) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:06,701::__init__::225::IOProcess::(_processLogs) (357) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,701::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:24:06,701::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:06,702::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:24:06,708::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000245846 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:06,708::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:06,708::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:06,708::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,708::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,708::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,709::__init__::225::IOProcess::(_processLogs) (358) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:06,709::__init__::225::IOProcess::(_processLogs) (358) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:06,709::__init__::225::IOProcess::(_processLogs) (358) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,710::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:06,710::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:06,710::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:24:06,710::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) (359) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) (359) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) (359) Building response >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) (359) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:06,711::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:06,712::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:24:06,712::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:06,712::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,712::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) (360) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) (360) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) (360) Building response >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) (360) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:24:06,713::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:06,714::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:06,714::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:06,714::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:06,714::__init__::225::IOProcess::(_processLogs) (361) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:06,715::__init__::225::IOProcess::(_processLogs) (361) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:06,715::__init__::225::IOProcess::(_processLogs) (361) Building response >Thread-17::DEBUG::2014-08-22 09:24:06,715::__init__::225::IOProcess::(_processLogs) (361) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:06,715::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:06,715::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:24:08,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:08,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121848 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:10,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:10,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115653 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:12,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:12,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115719 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:14,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:14,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117089 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:15,337::task::595::Storage.TaskManager.Task::(_updateState) Task=`96ef751e-8154-4793-a081-129983c7300f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:15,337::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:24:15,337::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205248', 'lastCheck': '8.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245846', 'lastCheck': '8.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:15,337::task::1191::Storage.TaskManager.Task::(prepare) Task=`96ef751e-8154-4793-a081-129983c7300f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205248', 'lastCheck': '8.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000245846', 'lastCheck': '8.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:15,337::task::595::Storage.TaskManager.Task::(_updateState) Task=`96ef751e-8154-4793-a081-129983c7300f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:15,337::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:15,337::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:15,337::task::993::Storage.TaskManager.Task::(_decref) Task=`96ef751e-8154-4793-a081-129983c7300f`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:24:15,618::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:24:15,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e46500d-139d-473f-8fd8-f447f417c625`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:15,618::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:24:15,618::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:15,618::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e46500d-139d-473f-8fd8-f447f417c625`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:15,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e46500d-139d-473f-8fd8-f447f417c625`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:15,618::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:15,619::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:15,619::task::993::Storage.TaskManager.Task::(_decref) Task=`8e46500d-139d-473f-8fd8-f447f417c625`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:24:15,661::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:24:15,661::task::595::Storage.TaskManager.Task::(_updateState) Task=`4436c109-a5b9-47f7-a92c-78a1f948ceb9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:24:15,661::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:24:15,661::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6c873ee6-13fc-4e6a-937d-25b9db9bb3ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:24:15,661::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:24:15,662::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:24:15,662::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6c873ee6-13fc-4e6a-937d-25b9db9bb3ae`::Granted request >Thread-46::DEBUG::2014-08-22 09:24:15,662::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4436c109-a5b9-47f7-a92c-78a1f948ceb9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:24:15,662::task::993::Storage.TaskManager.Task::(_decref) Task=`4436c109-a5b9-47f7-a92c-78a1f948ceb9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:24:15,664::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:15,664::task::1191::Storage.TaskManager.Task::(prepare) Task=`4436c109-a5b9-47f7-a92c-78a1f948ceb9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:15,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`4436c109-a5b9-47f7-a92c-78a1f948ceb9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:24:15,664::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:24:15,664::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:24:15,664::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:24:15,664::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:24:15,664::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:24:15,665::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:24:15,665::task::993::Storage.TaskManager.Task::(_decref) Task=`4436c109-a5b9-47f7-a92c-78a1f948ceb9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:24:16,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:16,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116353 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:16,513::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:16,513::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,513::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,513::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,513::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,513::__init__::225::IOProcess::(_processLogs) (362) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:16,514::__init__::225::IOProcess::(_processLogs) (362) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:16,514::__init__::225::IOProcess::(_processLogs) (362) Queuing response >Thread-21::DEBUG::2014-08-22 09:24:16,515::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:16,515::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,521::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:24:16,522::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000223395 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) (363) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:16,523::__init__::225::IOProcess::(_processLogs) (363) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:16,524::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(363) Building response >Thread-17::DEBUG::2014-08-22 09:24:16,525::__init__::225::IOProcess::(_processLogs) (363) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:16,525::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,525::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) (364) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:16,715::__init__::225::IOProcess::(_processLogs) (364) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:16,716::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(364) Building response >Thread-22::DEBUG::2014-08-22 09:24:16,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:16,717::__init__::225::IOProcess::(_processLogs) (364) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:16,723::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,723::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:24:16,724::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00018575 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) (365) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:16,725::__init__::225::IOProcess::(_processLogs) (365) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:16,726::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(365) Building response >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) (365) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,727::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,728::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,728::__init__::225::IOProcess::(_processLogs) (366) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:16,728::__init__::225::IOProcess::(_processLogs) (366) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:16,728::__init__::225::IOProcess::(_processLogs) (366) Building response >Thread-17::DEBUG::2014-08-22 09:24:16,728::__init__::225::IOProcess::(_processLogs) (366) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:16,728::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,729::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:16,729::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:16,729::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:24:16,729::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:16,729::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) (367) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) (367) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) (367) Building response >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) (367) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:16,730::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:16,731::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:24:16,731::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:16,731::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:16,731::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) (368) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) (368) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) (368) Building response >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) (368) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:16,732::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:24:18,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:18,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154058 s, 66.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:20,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:20,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175053 s, 58.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:22,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:22,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115782 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:24,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:24,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113811 s, 90.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:25,713::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:24:25,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f110fc2-306b-4fa7-b292-93ea73bffbb8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:25,714::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:24:25,714::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:25,714::task::1191::Storage.TaskManager.Task::(prepare) Task=`7f110fc2-306b-4fa7-b292-93ea73bffbb8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:25,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f110fc2-306b-4fa7-b292-93ea73bffbb8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:25,715::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:25,715::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:25,715::task::993::Storage.TaskManager.Task::(_decref) Task=`7f110fc2-306b-4fa7-b292-93ea73bffbb8`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:24:25,758::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:24:25,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1093863-a6c0-4ac1-8445-22b31a5cb682`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:24:25,759::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:24:25,759::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`54abc54b-693a-4f7a-ba13-54ed2add78d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:24:25,759::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:24:25,759::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:24:25,759::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`54abc54b-693a-4f7a-ba13-54ed2add78d3`::Granted request >Thread-46::DEBUG::2014-08-22 09:24:25,760::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b1093863-a6c0-4ac1-8445-22b31a5cb682`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:24:25,760::task::993::Storage.TaskManager.Task::(_decref) Task=`b1093863-a6c0-4ac1-8445-22b31a5cb682`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:24:25,762::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:25,762::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1093863-a6c0-4ac1-8445-22b31a5cb682`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:25,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1093863-a6c0-4ac1-8445-22b31a5cb682`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:24:25,762::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:24:25,762::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:24:25,762::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:24:25,763::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:24:25,763::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:24:25,763::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:24:25,763::task::993::Storage.TaskManager.Task::(_decref) Task=`b1093863-a6c0-4ac1-8445-22b31a5cb682`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:24:26,525::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,525::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:26,525::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,525::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,526::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,526::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,526::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,526::__init__::225::IOProcess::(_processLogs) (369) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:26,526::__init__::225::IOProcess::(_processLogs) (369) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:26,526::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(369) Building response >Thread-21::DEBUG::2014-08-22 09:24:26,527::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:26,527::__init__::225::IOProcess::(_processLogs) (369) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:24:26,534::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00022495 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:26,534::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:26,535::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) (370) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:26,536::__init__::225::IOProcess::(_processLogs) (370) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:26,537::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(370) Building response >Thread-17::DEBUG::2014-08-22 09:24:26,538::__init__::225::IOProcess::(_processLogs) (370) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:26,538::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:24:26,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:26,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117832 s, 86.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:26,732::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,732::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:26,732::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,732::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,733::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,733::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,733::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,733::__init__::225::IOProcess::(_processLogs) (371) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:26,733::__init__::225::IOProcess::(_processLogs) (371) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:26,733::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(371) Building response >Thread-22::DEBUG::2014-08-22 09:24:26,734::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:26,734::__init__::225::IOProcess::(_processLogs) (371) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,740::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:26,741::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:24:26,741::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000229362 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:26,742::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,742::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:26,742::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,743::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,743::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,743::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,743::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,743::__init__::225::IOProcess::(_processLogs) (372) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:26,743::__init__::225::IOProcess::(_processLogs) (372) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:26,744::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(372) Building response >Thread-17::DEBUG::2014-08-22 09:24:26,744::__init__::225::IOProcess::(_processLogs) (372) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,744::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:26,744::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:26,745::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,745::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:24:26,745::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,745::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,745::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) (373) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) (373) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) (373) Building response >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) (373) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,746::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:24:26,747::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,747::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,747::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,747::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,747::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,747::__init__::225::IOProcess::(_processLogs) (374) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) (374) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) (374) Building response >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) (374) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:24:26,748::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:26,749::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:26,749::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:26,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:26,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:26,749::__init__::225::IOProcess::(_processLogs) (375) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:26,749::__init__::225::IOProcess::(_processLogs) (375) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:26,750::__init__::225::IOProcess::(_processLogs) (375) Building response >Thread-17::DEBUG::2014-08-22 09:24:26,750::__init__::225::IOProcess::(_processLogs) (375) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:26,750::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:26,750::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:24:28,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:28,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018384 s, 55.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:30,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`10a58400-8ba8-49cb-bd7c-ffe7358fa6cb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:30,608::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:24:30,608::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00022495', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000229362', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:30,609::task::1191::Storage.TaskManager.Task::(prepare) Task=`10a58400-8ba8-49cb-bd7c-ffe7358fa6cb`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00022495', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000229362', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:30,609::task::595::Storage.TaskManager.Task::(_updateState) Task=`10a58400-8ba8-49cb-bd7c-ffe7358fa6cb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:30,609::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:30,609::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:30,609::task::993::Storage.TaskManager.Task::(_decref) Task=`10a58400-8ba8-49cb-bd7c-ffe7358fa6cb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:24:30,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:30,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116363 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:32,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:32,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183761 s, 55.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:34,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:34,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118416 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:35,875::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:24:35,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`e917a32a-8805-4886-945f-6b4eeb3042bc`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:35,876::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:24:35,876::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:35,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`e917a32a-8805-4886-945f-6b4eeb3042bc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:35,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`e917a32a-8805-4886-945f-6b4eeb3042bc`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:35,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:35,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:35,877::task::993::Storage.TaskManager.Task::(_decref) Task=`e917a32a-8805-4886-945f-6b4eeb3042bc`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:24:35,948::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:24:35,948::task::595::Storage.TaskManager.Task::(_updateState) Task=`d43f8be6-b0a6-4277-b0e8-be9f9f7a633c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:24:35,948::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:24:35,949::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d4c1da42-fba9-400b-b6fa-39fa5f80863b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:24:35,949::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:24:35,949::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:24:35,949::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d4c1da42-fba9-400b-b6fa-39fa5f80863b`::Granted request >Thread-46::DEBUG::2014-08-22 09:24:35,949::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d43f8be6-b0a6-4277-b0e8-be9f9f7a633c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:24:35,949::task::993::Storage.TaskManager.Task::(_decref) Task=`d43f8be6-b0a6-4277-b0e8-be9f9f7a633c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:24:35,951::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:35,951::task::1191::Storage.TaskManager.Task::(prepare) Task=`d43f8be6-b0a6-4277-b0e8-be9f9f7a633c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:35,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`d43f8be6-b0a6-4277-b0e8-be9f9f7a633c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:24:35,951::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:24:35,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:24:35,952::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:24:35,952::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:24:35,952::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:24:35,952::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:24:35,952::task::993::Storage.TaskManager.Task::(_decref) Task=`d43f8be6-b0a6-4277-b0e8-be9f9f7a633c`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:24:36,538::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,538::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:36,538::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,538::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,538::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) (376) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) (376) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(376) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) (376) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,539::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,540::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:24:36,540::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:24:36,546::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000215102 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:36,547::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,547::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:36,547::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,547::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,548::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,548::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,548::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,549::__init__::225::IOProcess::(_processLogs) (377) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:36,549::__init__::225::IOProcess::(_processLogs) (377) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:36,549::__init__::225::IOProcess::(_processLogs) (377) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,549::__init__::225::IOProcess::(_processLogs) (377) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,549::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,549::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) (378) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) (378) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) (378) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,750::__init__::225::IOProcess::(_processLogs) (378) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,751::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:24:36,752::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:24:36,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000238465 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:36,757::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,757::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) (379) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) (379) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) (379) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) (379) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,758::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,759::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:36,760::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,760::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:24:36,760::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,760::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) (380) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) (380) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) (380) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) (380) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:36,761::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,762::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:24:36,762::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,762::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,762::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,762::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,762::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) (381) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) (381) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) (381) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) (381) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:24:36,763::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:36,764::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:36,764::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:36,764::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) (382) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) (382) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) (382) Building response >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) (382) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:36,765::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:24:36,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:37,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118067 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:39,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:39,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115818 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:41,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:41,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115126 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:43,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:43,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115638 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:45,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:45,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115485 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:45,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3ac0c37-47ea-4daa-a9b5-a3aebc30eb8b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:45,853::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:24:45,853::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000215102', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000238465', 'lastCheck': '9.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:45,853::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3ac0c37-47ea-4daa-a9b5-a3aebc30eb8b`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000215102', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000238465', 'lastCheck': '9.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:24:45,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3ac0c37-47ea-4daa-a9b5-a3aebc30eb8b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:45,853::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:45,853::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:45,853::task::993::Storage.TaskManager.Task::(_decref) Task=`a3ac0c37-47ea-4daa-a9b5-a3aebc30eb8b`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:24:46,019::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:24:46,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`566504ab-37f0-4861-8615-b1042cbfdbe2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:46,019::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:24:46,019::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:46,020::task::1191::Storage.TaskManager.Task::(prepare) Task=`566504ab-37f0-4861-8615-b1042cbfdbe2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:46,020::task::595::Storage.TaskManager.Task::(_updateState) Task=`566504ab-37f0-4861-8615-b1042cbfdbe2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:46,020::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:46,020::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:46,020::task::993::Storage.TaskManager.Task::(_decref) Task=`566504ab-37f0-4861-8615-b1042cbfdbe2`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:24:46,063::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:24:46,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`47ae2f57-7465-4daf-badd-8a124b4d021c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:24:46,063::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:24:46,064::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`47abed0b-1e79-45ff-ba74-ffc20aff12e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:24:46,064::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:24:46,064::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:24:46,064::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`47abed0b-1e79-45ff-ba74-ffc20aff12e9`::Granted request >Thread-46::DEBUG::2014-08-22 09:24:46,064::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`47ae2f57-7465-4daf-badd-8a124b4d021c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:24:46,064::task::993::Storage.TaskManager.Task::(_decref) Task=`47ae2f57-7465-4daf-badd-8a124b4d021c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:24:46,066::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:46,066::task::1191::Storage.TaskManager.Task::(prepare) Task=`47ae2f57-7465-4daf-badd-8a124b4d021c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:46,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`47ae2f57-7465-4daf-badd-8a124b4d021c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:24:46,066::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:24:46,067::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:24:46,067::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:24:46,067::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:24:46,067::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:24:46,067::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:24:46,067::task::993::Storage.TaskManager.Task::(_decref) Task=`47ae2f57-7465-4daf-badd-8a124b4d021c`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) (383) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:46,549::__init__::225::IOProcess::(_processLogs) (383) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:46,550::__init__::225::IOProcess::(_processLogs) (383) Queuing response >Thread-21::DEBUG::2014-08-22 09:24:46,551::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:46,551::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,557::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:24:46,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00017533 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:46,558::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:46,558::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:46,559::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,559::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,559::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,559::__init__::225::IOProcess::(_processLogs) (384) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:46,559::__init__::225::IOProcess::(_processLogs) (384) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:46,560::__init__::225::IOProcess::(_processLogs) (384) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:46,560::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,560::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) (385) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:46,765::__init__::225::IOProcess::(_processLogs) (385) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:46,766::__init__::225::IOProcess::(_processLogs) (385) Queuing response >Thread-22::DEBUG::2014-08-22 09:24:46,767::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:46,767::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,772::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:24:46,773::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206731 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) (386) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:46,774::__init__::225::IOProcess::(_processLogs) (386) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:46,775::__init__::225::IOProcess::(_processLogs) (386) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:46,776::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,776::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:46,776::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:24:46,776::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) (387) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) (387) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) (387) Building response >Thread-17::DEBUG::2014-08-22 09:24:46,777::__init__::225::IOProcess::(_processLogs) (387) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:46,778::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,778::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:46,778::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:24:46,778::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:46,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) (388) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) (388) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) (388) Building response >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) (388) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:46,779::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:24:46,780::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:46,780::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:46,780::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) (389) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) (389) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) (389) Building response >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) (389) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:46,781::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:24:47,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:47,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118645 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:49,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:49,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114661 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:51,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:51,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116234 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:53,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:53,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113507 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:55,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:55,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116461 s, 87.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:24:56,144::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:24:56,144::task::595::Storage.TaskManager.Task::(_updateState) Task=`3514c692-f549-4443-a32b-c9f2e7dcf1ed`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:24:56,145::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:24:56,145::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:56,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`3514c692-f549-4443-a32b-c9f2e7dcf1ed`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:24:56,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`3514c692-f549-4443-a32b-c9f2e7dcf1ed`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:24:56,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:24:56,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:24:56,145::task::993::Storage.TaskManager.Task::(_decref) Task=`3514c692-f549-4443-a32b-c9f2e7dcf1ed`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:24:56,175::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:24:56,175::task::595::Storage.TaskManager.Task::(_updateState) Task=`57511e5c-224a-4e29-9b7f-e80e845b4f3f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:24:56,176::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:24:56,176::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23860f20-ae77-4c92-be43-6fbd248dfa0d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:24:56,176::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:24:56,176::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:24:56,176::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23860f20-ae77-4c92-be43-6fbd248dfa0d`::Granted request >Thread-46::DEBUG::2014-08-22 09:24:56,176::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`57511e5c-224a-4e29-9b7f-e80e845b4f3f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:24:56,176::task::993::Storage.TaskManager.Task::(_decref) Task=`57511e5c-224a-4e29-9b7f-e80e845b4f3f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:24:56,178::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:56,178::task::1191::Storage.TaskManager.Task::(prepare) Task=`57511e5c-224a-4e29-9b7f-e80e845b4f3f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:24:56,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`57511e5c-224a-4e29-9b7f-e80e845b4f3f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:24:56,179::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:24:56,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:24:56,179::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:24:56,179::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:24:56,179::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:24:56,179::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:24:56,179::task::993::Storage.TaskManager.Task::(_decref) Task=`57511e5c-224a-4e29-9b7f-e80e845b4f3f`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) (390) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:56,561::__init__::225::IOProcess::(_processLogs) (390) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:56,562::__init__::225::IOProcess::(_processLogs) (390) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,563::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:24:56,563::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:56,563::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:24:56,569::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182932 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) (391) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:56,570::__init__::225::IOProcess::(_processLogs) (391) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:56,571::__init__::225::IOProcess::(_processLogs) (391) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,572::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:56,572::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) (392) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:56,781::__init__::225::IOProcess::(_processLogs) (392) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:56,782::__init__::225::IOProcess::(_processLogs) (392) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,782::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:24:56,783::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:24:56,783::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:24:56,789::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218107 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:24:56,789::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:24:56,789::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:24:56,789::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,790::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,790::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,790::__init__::225::IOProcess::(_processLogs) (393) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:24:56,790::__init__::225::IOProcess::(_processLogs) (393) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:24:56,791::__init__::225::IOProcess::(_processLogs) (393) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,791::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:56,791::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:24:56,791::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:24:56,791::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:56,792::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,792::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,792::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,792::__init__::225::IOProcess::(_processLogs) (394) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:56,792::__init__::225::IOProcess::(_processLogs) (394) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:56,792::__init__::225::IOProcess::(_processLogs) (394) Building response >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) (394) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) (395) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:56,793::__init__::225::IOProcess::(_processLogs) (395) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) (395) Building response >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) (395) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:24:56,794::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:24:56,795::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:24:56,795::__init__::225::IOProcess::(_processLogs) (396) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:24:56,795::__init__::225::IOProcess::(_processLogs) (396) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:24:56,795::__init__::225::IOProcess::(_processLogs) (396) Building response >Thread-17::DEBUG::2014-08-22 09:24:56,795::__init__::225::IOProcess::(_processLogs) (396) Queuing response >Thread-17::DEBUG::2014-08-22 09:24:56,795::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:24:57,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:57,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119184 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:24:59,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:24:59,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119209 s, 85.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:25:01,061::task::595::Storage.TaskManager.Task::(_updateState) Task=`03e1cb50-253c-474d-a37f-08c61cbe20f4`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:01,061::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:25:01,061::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182932', 'lastCheck': '4.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218107', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:25:01,061::task::1191::Storage.TaskManager.Task::(prepare) Task=`03e1cb50-253c-474d-a37f-08c61cbe20f4`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182932', 'lastCheck': '4.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218107', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:25:01,062::task::595::Storage.TaskManager.Task::(_updateState) Task=`03e1cb50-253c-474d-a37f-08c61cbe20f4`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:01,062::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:01,062::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:01,062::task::993::Storage.TaskManager.Task::(_decref) Task=`03e1cb50-253c-474d-a37f-08c61cbe20f4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:25:01,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:01,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116894 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:03,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:03,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013116 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:05,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:05,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117076 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:25:06,253::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:25:06,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc8119ca-73f2-409f-b4b2-cae87be64985`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:06,253::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:25:06,254::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:06,254::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc8119ca-73f2-409f-b4b2-cae87be64985`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:06,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc8119ca-73f2-409f-b4b2-cae87be64985`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:06,254::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:06,254::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:06,254::task::993::Storage.TaskManager.Task::(_decref) Task=`fc8119ca-73f2-409f-b4b2-cae87be64985`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:25:06,309::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:25:06,309::task::595::Storage.TaskManager.Task::(_updateState) Task=`0abcd24f-e6ed-4b10-912d-13122a4cd8a8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:25:06,309::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:25:06,310::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ad489814-1034-4581-8def-c5b29a7bb86b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:25:06,310::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:25:06,310::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:25:06,310::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ad489814-1034-4581-8def-c5b29a7bb86b`::Granted request >Thread-46::DEBUG::2014-08-22 09:25:06,310::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0abcd24f-e6ed-4b10-912d-13122a4cd8a8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:25:06,310::task::993::Storage.TaskManager.Task::(_decref) Task=`0abcd24f-e6ed-4b10-912d-13122a4cd8a8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:25:06,313::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:06,313::task::1191::Storage.TaskManager.Task::(prepare) Task=`0abcd24f-e6ed-4b10-912d-13122a4cd8a8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:06,313::task::595::Storage.TaskManager.Task::(_updateState) Task=`0abcd24f-e6ed-4b10-912d-13122a4cd8a8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:25:06,313::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:25:06,313::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:06,314::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:25:06,314::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:25:06,314::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:25:06,314::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:25:06,314::task::993::Storage.TaskManager.Task::(_decref) Task=`0abcd24f-e6ed-4b10-912d-13122a4cd8a8`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:25:06,572::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:06,572::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:06,572::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,573::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,573::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,573::__init__::225::IOProcess::(_processLogs) (397) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:06,573::__init__::225::IOProcess::(_processLogs) (397) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:06,573::__init__::225::IOProcess::(_processLogs) (397) Queuing response >Thread-21::DEBUG::2014-08-22 09:25:06,574::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:06,574::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:06,580::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:25:06,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00311433 s, 127 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) (398) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:06,597::__init__::225::IOProcess::(_processLogs) (398) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:06,600::__init__::225::IOProcess::(_processLogs) (398) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:06,600::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:06,796::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:06,796::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:06,796::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,796::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,796::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,797::__init__::225::IOProcess::(_processLogs) (399) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:06,797::__init__::225::IOProcess::(_processLogs) (399) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:06,806::__init__::225::IOProcess::(_processLogs) (399) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:06,807::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:06,807::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:25:06,807::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:25:06,821::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000429399 s, 1.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) (400) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:06,822::__init__::225::IOProcess::(_processLogs) (400) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:06,825::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:25:06,825::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:06,825::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,825::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) (401) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) (401) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) (401) Building response >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) (401) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:25:06,826::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) (402) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) (402) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) (402) Building response >Thread-17::DEBUG::2014-08-22 09:25:06,827::__init__::225::IOProcess::(_processLogs) (402) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:06,828::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:06,828::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:06,828::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:25:06,828::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:06,828::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:06,828::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:06,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:06,829::__init__::225::IOProcess::(_processLogs) (403) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:06,829::__init__::225::IOProcess::(_processLogs) (403) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:06,829::__init__::225::IOProcess::(_processLogs) (403) Building response >Thread-17::DEBUG::2014-08-22 09:25:06,829::__init__::225::IOProcess::(_processLogs) (403) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:06,829::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:06,830::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:25:07,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:07,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120319 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:09,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:09,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116904 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:11,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:11,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139245 s, 73.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:13,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:13,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115844 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:15,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:15,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121339 s, 84.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:25:16,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6fd842e-e52f-4946-9491-9600d721b58f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:16,349::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:25:16,349::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00311433', 'lastCheck': '9.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429399', 'lastCheck': '9.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:25:16,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`c6fd842e-e52f-4946-9491-9600d721b58f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00311433', 'lastCheck': '9.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429399', 'lastCheck': '9.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:25:16,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6fd842e-e52f-4946-9491-9600d721b58f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:16,349::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:16,349::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:16,349::task::993::Storage.TaskManager.Task::(_decref) Task=`c6fd842e-e52f-4946-9491-9600d721b58f`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:25:16,390::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [28d77049] >Thread-47::DEBUG::2014-08-22 09:25:16,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`3277fbe2-8808-4f5d-8bc7-e9477788211a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:16,391::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:25:16,391::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:16,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`3277fbe2-8808-4f5d-8bc7-e9477788211a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:16,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`3277fbe2-8808-4f5d-8bc7-e9477788211a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:16,391::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:16,391::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:16,391::task::993::Storage.TaskManager.Task::(_decref) Task=`3277fbe2-8808-4f5d-8bc7-e9477788211a`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:25:16,484::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [28d77049] >Thread-46::DEBUG::2014-08-22 09:25:16,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`018dab96-1499-48e7-93e9-33672528fef2`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:25:16,484::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:25:16,484::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a7a7b449-8f70-4d0b-99dd-2b5bf7a9ebb7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:25:16,484::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:25:16,484::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:25:16,484::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a7a7b449-8f70-4d0b-99dd-2b5bf7a9ebb7`::Granted request >Thread-46::DEBUG::2014-08-22 09:25:16,485::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`018dab96-1499-48e7-93e9-33672528fef2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:25:16,485::task::993::Storage.TaskManager.Task::(_decref) Task=`018dab96-1499-48e7-93e9-33672528fef2`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:25:16,487::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:16,487::task::1191::Storage.TaskManager.Task::(prepare) Task=`018dab96-1499-48e7-93e9-33672528fef2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:16,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`018dab96-1499-48e7-93e9-33672528fef2`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:25:16,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:25:16,487::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:16,487::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:25:16,487::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:25:16,487::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:25:16,487::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:25:16,488::task::993::Storage.TaskManager.Task::(_decref) Task=`018dab96-1499-48e7-93e9-33672528fef2`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) (404) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:16,602::__init__::225::IOProcess::(_processLogs) (404) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:16,603::__init__::225::IOProcess::(_processLogs) (404) Queuing response >Thread-21::DEBUG::2014-08-22 09:25:16,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:16,603::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,609::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:25:16,610::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00020758 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) (405) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:16,611::__init__::225::IOProcess::(_processLogs) (405) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:16,612::__init__::225::IOProcess::(_processLogs) (405) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:16,613::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,613::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) (406) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) (406) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:16,830::__init__::225::IOProcess::(_processLogs) (406) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:16,831::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,831::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:25:16,831::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:25:16,837::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000256228 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) (407) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:16,838::__init__::225::IOProcess::(_processLogs) (407) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:16,839::__init__::225::IOProcess::(_processLogs) (407) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:16,839::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,839::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) (408) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:16,840::__init__::225::IOProcess::(_processLogs) (408) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:16,841::__init__::225::IOProcess::(_processLogs) (408) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:16,841::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,841::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:16,841::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:25:16,841::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) (409) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) (409) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) (409) Building response >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) (409) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:16,842::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,843::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:16,843::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:25:16,843::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:16,843::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:16,843::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) (410) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) (410) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) (410) Building response >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) (410) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:16,844::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:25:17,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:18,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119028 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:20,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:20,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116958 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:22,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:22,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118676 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:24,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:24,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115438 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:26,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:26,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012036 s, 85.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:25:26,536::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:25:26,537::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c6e923e-3103-458c-baf3-93474706bcb7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:26,537::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:25:26,537::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:26,537::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c6e923e-3103-458c-baf3-93474706bcb7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:26,537::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c6e923e-3103-458c-baf3-93474706bcb7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:26,537::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:26,537::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:26,538::task::993::Storage.TaskManager.Task::(_decref) Task=`4c6e923e-3103-458c-baf3-93474706bcb7`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:25:26,570::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:25:26,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8c7aa28-6716-40d5-8e32-0d9eaee0f776`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:25:26,571::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:25:26,571::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0ec8f432-1d71-4168-9d6b-f56883f2e575`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:25:26,571::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:25:26,571::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:25:26,571::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0ec8f432-1d71-4168-9d6b-f56883f2e575`::Granted request >Thread-46::DEBUG::2014-08-22 09:25:26,572::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d8c7aa28-6716-40d5-8e32-0d9eaee0f776`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:25:26,572::task::993::Storage.TaskManager.Task::(_decref) Task=`d8c7aa28-6716-40d5-8e32-0d9eaee0f776`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:25:26,574::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:26,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`d8c7aa28-6716-40d5-8e32-0d9eaee0f776`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:26,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8c7aa28-6716-40d5-8e32-0d9eaee0f776`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:25:26,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:25:26,574::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:26,574::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:25:26,574::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:25:26,574::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:25:26,574::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:25:26,575::task::993::Storage.TaskManager.Task::(_decref) Task=`d8c7aa28-6716-40d5-8e32-0d9eaee0f776`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:25:26,613::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:26,613::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:26,614::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,614::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,614::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,614::__init__::225::IOProcess::(_processLogs) (411) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:26,614::__init__::225::IOProcess::(_processLogs) (411) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:26,614::__init__::225::IOProcess::(_processLogs) (411) Queuing response >Thread-21::DEBUG::2014-08-22 09:25:26,615::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:26,615::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,621::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:25:26,622::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000194269 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) (412) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:26,623::__init__::225::IOProcess::(_processLogs) (412) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:26,624::__init__::225::IOProcess::(_processLogs) (412) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:26,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,625::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:26,844::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:26,844::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:26,844::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,844::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,845::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,845::__init__::225::IOProcess::(_processLogs) (413) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:26,845::__init__::225::IOProcess::(_processLogs) (413) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:26,845::__init__::225::IOProcess::(_processLogs) (413) Queuing response >Thread-22::DEBUG::2014-08-22 09:25:26,846::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:26,846::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,852::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:25:26,853::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198833 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:26,853::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:26,853::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:26,853::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,853::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,854::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,854::__init__::225::IOProcess::(_processLogs) (414) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:26,854::__init__::225::IOProcess::(_processLogs) (414) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:26,854::__init__::225::IOProcess::(_processLogs) (414) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:26,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,855::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:26,855::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:25:26,855::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) (415) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) (415) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) (415) Building response >Thread-17::DEBUG::2014-08-22 09:25:26,856::__init__::225::IOProcess::(_processLogs) (415) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:26,857::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,857::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:26,857::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:25:26,857::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:26,857::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) (416) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) (416) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) (416) Building response >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) (416) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:25:26,858::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:26,859::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:26,859::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:26,859::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:26,860::__init__::225::IOProcess::(_processLogs) (417) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:26,860::__init__::225::IOProcess::(_processLogs) (417) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:26,860::__init__::225::IOProcess::(_processLogs) (417) Building response >Thread-17::DEBUG::2014-08-22 09:25:26,860::__init__::225::IOProcess::(_processLogs) (417) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:26,860::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:26,860::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:25:28,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:28,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123157 s, 83.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:30,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:30,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116369 s, 88.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:25:31,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`56d5869f-ba9e-47bf-8396-2477f5591fbe`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:31,583::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:25:31,583::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000194269', 'lastCheck': '5.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198833', 'lastCheck': '4.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:25:31,584::task::1191::Storage.TaskManager.Task::(prepare) Task=`56d5869f-ba9e-47bf-8396-2477f5591fbe`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000194269', 'lastCheck': '5.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198833', 'lastCheck': '4.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:25:31,584::task::595::Storage.TaskManager.Task::(_updateState) Task=`56d5869f-ba9e-47bf-8396-2477f5591fbe`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:31,584::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:31,584::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:31,584::task::993::Storage.TaskManager.Task::(_decref) Task=`56d5869f-ba9e-47bf-8396-2477f5591fbe`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:25:32,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:32,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217633 s, 47.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:34,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:34,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117936 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:36,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:36,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118091 s, 86.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:25:36,624::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:25:36,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`68df33c6-6d64-402c-9d96-e16373d7918e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:36,626::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:25:36,626::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:36,627::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-47::INFO::2014-08-22 09:25:36,627::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-17::DEBUG::2014-08-22 09:25:36,628::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-47::DEBUG::2014-08-22 09:25:36,628::task::1191::Storage.TaskManager.Task::(prepare) Task=`68df33c6-6d64-402c-9d96-e16373d7918e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-17::DEBUG::2014-08-22 09:25:36,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-47::DEBUG::2014-08-22 09:25:36,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`68df33c6-6d64-402c-9d96-e16373d7918e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:36,629::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:36,629::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-08-22 09:25:36,630::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-47::DEBUG::2014-08-22 09:25:36,630::task::993::Storage.TaskManager.Task::(_decref) Task=`68df33c6-6d64-402c-9d96-e16373d7918e`::ref 0 aborting False >Thread-21::DEBUG::2014-08-22 09:25:36,631::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:36,632::__init__::225::IOProcess::(_processLogs) (418) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:36,638::__init__::225::IOProcess::(_processLogs) (418) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:36,640::__init__::225::IOProcess::(_processLogs) (418) Building response >Thread-21::DEBUG::2014-08-22 09:25:36,640::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000241174 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:36,641::__init__::225::IOProcess::(_processLogs) (418) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,641::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:36,641::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) (419) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:36,642::__init__::225::IOProcess::(_processLogs) (419) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:36,643::__init__::225::IOProcess::(_processLogs) (419) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,643::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:36,644::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-46::DEBUG::2014-08-22 09:25:36,675::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:25:36,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bf13cc2-2644-4491-a8fb-87c2f47ff9ad`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:25:36,675::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:25:36,675::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`95932724-70f9-418c-896a-d86e55b899c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:25:36,676::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:25:36,676::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:25:36,676::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`95932724-70f9-418c-896a-d86e55b899c5`::Granted request >Thread-46::DEBUG::2014-08-22 09:25:36,676::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4bf13cc2-2644-4491-a8fb-87c2f47ff9ad`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:25:36,676::task::993::Storage.TaskManager.Task::(_decref) Task=`4bf13cc2-2644-4491-a8fb-87c2f47ff9ad`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:25:36,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:36,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bf13cc2-2644-4491-a8fb-87c2f47ff9ad`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:36,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bf13cc2-2644-4491-a8fb-87c2f47ff9ad`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:25:36,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:25:36,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:36,679::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:25:36,679::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:25:36,679::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:25:36,679::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:25:36,679::task::993::Storage.TaskManager.Task::(_decref) Task=`4bf13cc2-2644-4491-a8fb-87c2f47ff9ad`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) (420) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:36,860::__init__::225::IOProcess::(_processLogs) (420) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:36,861::__init__::225::IOProcess::(_processLogs) (420) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,861::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:25:36,861::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:36,862::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:25:36,867::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000222946 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:36,868::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:36,868::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:36,868::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:36,868::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:36,868::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:36,869::__init__::225::IOProcess::(_processLogs) (421) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:36,869::__init__::225::IOProcess::(_processLogs) (421) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:36,869::__init__::225::IOProcess::(_processLogs) (421) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,870::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:36,870::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:36,870::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:25:36,870::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) (422) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) (422) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) (422) Building response >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) (422) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,871::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:36,872::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:36,872::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:25:36,872::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:36,872::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:36,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) (423) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) (423) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) (423) Building response >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) (423) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:25:36,873::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:36,874::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:36,874::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:36,874::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:36,875::__init__::225::IOProcess::(_processLogs) (424) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:36,875::__init__::225::IOProcess::(_processLogs) (424) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:36,875::__init__::225::IOProcess::(_processLogs) (424) Building response >Thread-17::DEBUG::2014-08-22 09:25:36,875::__init__::225::IOProcess::(_processLogs) (424) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:36,875::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:36,875::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:25:38,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:38,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119959 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:40,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:40,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115367 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:42,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:42,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115448 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:44,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:44,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114503 s, 89.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:46,644::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:46,644::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:46,644::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:46,644::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,644::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,645::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,645::__init__::225::IOProcess::(_processLogs) (425) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:46,645::__init__::225::IOProcess::(_processLogs) (425) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:46,645::__init__::225::IOProcess::(_processLogs) (425) Queuing response >Thread-21::DEBUG::2014-08-22 09:25:46,646::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:46,646::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,652::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:25:46,653::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000220874 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) (426) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:46,654::__init__::225::IOProcess::(_processLogs) (426) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:46,655::__init__::225::IOProcess::(_processLogs) (426) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:46,656::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,656::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-47::DEBUG::2014-08-22 09:25:46,745::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:25:46,746::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c40b215-0a4a-40df-aafa-258f8b05841d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:46,746::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:25:46,746::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:46,746::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c40b215-0a4a-40df-aafa-258f8b05841d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:46,746::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c40b215-0a4a-40df-aafa-258f8b05841d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:46,746::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:46,746::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:46,746::task::993::Storage.TaskManager.Task::(_decref) Task=`5c40b215-0a4a-40df-aafa-258f8b05841d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:25:46,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:46,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119019 s, 86.0 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:25:46,797::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:25:46,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fd2a949-39bd-4a74-be29-2d80e408072c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:25:46,801::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::DEBUG::2014-08-22 09:25:46,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01efadc-a6cc-468b-919b-ab72b90bd23e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:25:46,804::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-46::DEBUG::2014-08-22 09:25:46,804::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6a5f1db9-551f-4e3d-87ed-466e7410d86b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:25:46,805::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-47::INFO::2014-08-22 09:25:46,806::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000220874', 'lastCheck': '0.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000222946', 'lastCheck': '9.9', 'valid': True}} >Thread-46::DEBUG::2014-08-22 09:25:46,807::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-47::DEBUG::2014-08-22 09:25:46,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`a01efadc-a6cc-468b-919b-ab72b90bd23e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000220874', 'lastCheck': '0.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000222946', 'lastCheck': '9.9', 'valid': True}} >Thread-46::DEBUG::2014-08-22 09:25:46,808::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6a5f1db9-551f-4e3d-87ed-466e7410d86b`::Granted request >Thread-47::DEBUG::2014-08-22 09:25:46,808::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01efadc-a6cc-468b-919b-ab72b90bd23e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:46,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:46,811::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:46,811::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3fd2a949-39bd-4a74-be29-2d80e408072c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-47::DEBUG::2014-08-22 09:25:46,812::task::993::Storage.TaskManager.Task::(_decref) Task=`a01efadc-a6cc-468b-919b-ab72b90bd23e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:25:46,812::task::993::Storage.TaskManager.Task::(_decref) Task=`3fd2a949-39bd-4a74-be29-2d80e408072c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:25:46,836::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:46,837::task::1191::Storage.TaskManager.Task::(prepare) Task=`3fd2a949-39bd-4a74-be29-2d80e408072c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:46,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fd2a949-39bd-4a74-be29-2d80e408072c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:25:46,838::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:25:46,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:46,841::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:25:46,843::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:25:46,844::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:25:46,844::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:25:46,845::task::993::Storage.TaskManager.Task::(_decref) Task=`3fd2a949-39bd-4a74-be29-2d80e408072c`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) (427) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:46,875::__init__::225::IOProcess::(_processLogs) (427) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:46,876::__init__::225::IOProcess::(_processLogs) (427) Queuing response >Thread-22::DEBUG::2014-08-22 09:25:46,877::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:46,877::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,883::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:25:46,884::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000186177 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:46,884::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) (428) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:46,885::__init__::225::IOProcess::(_processLogs) (428) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:46,886::__init__::225::IOProcess::(_processLogs) (428) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:46,886::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,886::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:46,887::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:25:46,887::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:46,887::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,887::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) (429) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) (429) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) (429) Building response >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) (429) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:25:46,888::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) (430) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) (430) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) (430) Building response >Thread-17::DEBUG::2014-08-22 09:25:46,889::__init__::225::IOProcess::(_processLogs) (430) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:46,890::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,890::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:46,890::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:25:46,890::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:46,890::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) (431) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) (431) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) (431) Building response >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) (431) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:46,891::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:25:48,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:48,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118278 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:50,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:50,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120528 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:52,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:52,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154785 s, 66.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:55,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:55,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118797 s, 86.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:56,656::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:56,656::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:56,657::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:25:56,658::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:56,658::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,664::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:25:56,665::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000208712 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:56,665::__init__::225::IOProcess::(_processLogs) (432) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:56,666::__init__::225::IOProcess::(_processLogs) (432) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:56,666::__init__::225::IOProcess::(_processLogs) (432) Building response >Thread-17::DEBUG::2014-08-22 09:25:56,666::__init__::225::IOProcess::(_processLogs) (432) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:56,666::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:56,666::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:56,666::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:25:56,667::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:56,667::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:56,667::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,667::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:56,667::__init__::225::IOProcess::(_processLogs) (433) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:56,667::__init__::225::IOProcess::(_processLogs) (433) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:56,668::__init__::225::IOProcess::(_processLogs) (433) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:56,668::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:56,668::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:56,891::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:56,891::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:56,892::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:56,892::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,892::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:56,892::__init__::225::IOProcess::(_processLogs) (434) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:56,892::__init__::225::IOProcess::(_processLogs) (434) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:56,892::__init__::225::IOProcess::(_processLogs) (434) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:56,893::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:25:56,893::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:25:56,893::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:25:56,899::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021598 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:25:56,899::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:25:56,900::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:25:56,900::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:56,900::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,900::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:56,900::__init__::225::IOProcess::(_processLogs) (435) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:25:56,900::__init__::225::IOProcess::(_processLogs) (435) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:25:56,901::__init__::225::IOProcess::(_processLogs) (435) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:56,901::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:56,901::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:25:56,902::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:25:56,902::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:56,902::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:56,902::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,902::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) (436) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) (436) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) (436) Building response >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) (436) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:56,903::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,904::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:56,904::__init__::225::IOProcess::(_processLogs) (437) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:25:56,904::__init__::225::IOProcess::(_processLogs) (437) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:25:56,904::__init__::225::IOProcess::(_processLogs) (437) Queuing response >Thread-17::DEBUG::2014-08-22 09:25:56,904::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:56,905::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:25:56,905::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:25:56,905::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:25:56,905::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:25:56,905::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:25:56,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:25:56,906::__init__::225::IOProcess::(_processLogs) (438) Finding callback 'access'... >Thread-47::DEBUG::2014-08-22 09:25:56,906::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-17::DEBUG::2014-08-22 09:25:56,907::__init__::225::IOProcess::(_processLogs) (438) Got request for method 'access' >Thread-47::DEBUG::2014-08-22 09:25:56,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`faa74bca-fd47-4592-8e12-aeddd2ff6a01`::moving from state init -> state preparing >Thread-17::DEBUG::2014-08-22 09:25:56,908::__init__::225::IOProcess::(_processLogs) (438) Queuing response >Thread-47::INFO::2014-08-22 09:25:56,909::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:25:56,909::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:25:56,910::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::INFO::2014-08-22 09:25:56,910::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:56,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`faa74bca-fd47-4592-8e12-aeddd2ff6a01`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:25:56,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`faa74bca-fd47-4592-8e12-aeddd2ff6a01`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:25:56,911::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:25:56,911::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:25:56,911::task::993::Storage.TaskManager.Task::(_decref) Task=`faa74bca-fd47-4592-8e12-aeddd2ff6a01`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:25:56,974::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:25:56,974::task::595::Storage.TaskManager.Task::(_updateState) Task=`15ae61c1-34dd-4ef5-9081-258123082842`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:25:56,974::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:25:56,975::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9782940e-450b-493d-93aa-557ceecc3dbe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:25:56,975::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:25:56,975::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:25:56,975::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9782940e-450b-493d-93aa-557ceecc3dbe`::Granted request >Thread-46::DEBUG::2014-08-22 09:25:56,975::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`15ae61c1-34dd-4ef5-9081-258123082842`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:25:56,975::task::993::Storage.TaskManager.Task::(_decref) Task=`15ae61c1-34dd-4ef5-9081-258123082842`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:25:56,977::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:56,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`15ae61c1-34dd-4ef5-9081-258123082842`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:25:56,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`15ae61c1-34dd-4ef5-9081-258123082842`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:25:56,978::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:25:56,978::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:25:56,978::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:25:56,978::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:25:56,978::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:25:56,978::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:25:56,978::task::993::Storage.TaskManager.Task::(_decref) Task=`15ae61c1-34dd-4ef5-9081-258123082842`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:25:57,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:57,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012262 s, 83.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:25:59,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:25:59,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117076 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:01,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:01,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116742 s, 87.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:26:02,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e7ad3e8-70ed-4b3c-9d35-016ce46f2b82`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:02,046::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:26:02,047::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208712', 'lastCheck': '5.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021598', 'lastCheck': '5.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:02,048::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e7ad3e8-70ed-4b3c-9d35-016ce46f2b82`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208712', 'lastCheck': '5.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021598', 'lastCheck': '5.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:02,048::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e7ad3e8-70ed-4b3c-9d35-016ce46f2b82`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:02,049::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:02,049::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:02,050::task::993::Storage.TaskManager.Task::(_decref) Task=`9e7ad3e8-70ed-4b3c-9d35-016ce46f2b82`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:03,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:03,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117711 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:05,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:05,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116835 s, 87.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:06,669::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:06,669::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:06,669::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:06,669::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,669::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,669::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,670::__init__::225::IOProcess::(_processLogs) (439) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:06,670::__init__::225::IOProcess::(_processLogs) (439) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:06,670::__init__::225::IOProcess::(_processLogs) (439) Queuing response >Thread-21::DEBUG::2014-08-22 09:26:06,671::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:06,671::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,677::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:26:06,678::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000172731 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) (440) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:06,679::__init__::225::IOProcess::(_processLogs) (440) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:06,680::__init__::225::IOProcess::(_processLogs) (440) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:06,681::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,681::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) (441) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:06,910::__init__::225::IOProcess::(_processLogs) (441) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:06,911::__init__::225::IOProcess::(_processLogs) (441) Queuing response >Thread-22::DEBUG::2014-08-22 09:26:06,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:06,912::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,917::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:26:06,918::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000242069 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) (442) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:06,919::__init__::225::IOProcess::(_processLogs) (442) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:06,920::__init__::225::IOProcess::(_processLogs) (442) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:06,921::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,921::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:06,921::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:26:06,921::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:06,921::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) (443) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) (443) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) (443) Building response >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) (443) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:26:06,922::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:06,923::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,923::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,923::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,923::__init__::225::IOProcess::(_processLogs) (444) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:06,923::__init__::225::IOProcess::(_processLogs) (444) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:06,923::__init__::225::IOProcess::(_processLogs) (444) Building response >Thread-17::DEBUG::2014-08-22 09:26:06,924::__init__::225::IOProcess::(_processLogs) (444) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:06,924::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,924::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:06,924::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:26:06,924::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:06,924::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) (445) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) (445) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) (445) Building response >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) (445) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:06,925::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:06,926::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:26:07,035::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:26:07,036::task::595::Storage.TaskManager.Task::(_updateState) Task=`39cba550-80f5-42fd-b25c-215cd472ecad`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:07,036::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:26:07,036::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:07,036::task::1191::Storage.TaskManager.Task::(prepare) Task=`39cba550-80f5-42fd-b25c-215cd472ecad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:07,036::task::595::Storage.TaskManager.Task::(_updateState) Task=`39cba550-80f5-42fd-b25c-215cd472ecad`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:07,036::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:07,036::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:07,037::task::993::Storage.TaskManager.Task::(_decref) Task=`39cba550-80f5-42fd-b25c-215cd472ecad`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:26:07,084::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:26:07,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`46c8d123-723b-4ea2-bdc9-69c12b4ee63b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:26:07,084::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:26:07,084::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c0ca3ea8-e714-4ab4-8723-39eb67e0d3b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:26:07,084::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:26:07,084::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:26:07,085::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c0ca3ea8-e714-4ab4-8723-39eb67e0d3b1`::Granted request >Thread-46::DEBUG::2014-08-22 09:26:07,085::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`46c8d123-723b-4ea2-bdc9-69c12b4ee63b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:26:07,085::task::993::Storage.TaskManager.Task::(_decref) Task=`46c8d123-723b-4ea2-bdc9-69c12b4ee63b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:26:07,087::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:07,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`46c8d123-723b-4ea2-bdc9-69c12b4ee63b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:07,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`46c8d123-723b-4ea2-bdc9-69c12b4ee63b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:26:07,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:26:07,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:26:07,087::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:26:07,087::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:26:07,087::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:26:07,088::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:26:07,088::task::993::Storage.TaskManager.Task::(_decref) Task=`46c8d123-723b-4ea2-bdc9-69c12b4ee63b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:07,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:07,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123784 s, 82.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:09,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:09,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118552 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:11,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:11,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117005 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:13,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:13,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117906 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:15,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:15,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116967 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:16,681::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:16,681::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:16,682::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:16,682::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:16,682::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,682::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,682::__init__::225::IOProcess::(_processLogs) (446) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:16,682::__init__::225::IOProcess::(_processLogs) (446) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:16,683::__init__::225::IOProcess::(_processLogs) (446) Queuing response >Thread-21::DEBUG::2014-08-22 09:26:16,683::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:16,684::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,689::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:26:16,691::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000638742 s, 622 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:16,691::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:16,691::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:16,692::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:16,692::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,692::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,692::__init__::225::IOProcess::(_processLogs) (447) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:16,692::__init__::225::IOProcess::(_processLogs) (447) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:16,693::__init__::225::IOProcess::(_processLogs) (447) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:16,693::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,694::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:16,925::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) (448) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) (448) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:16,926::__init__::225::IOProcess::(_processLogs) (448) Queuing response >Thread-22::DEBUG::2014-08-22 09:26:16,927::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:16,927::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,932::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:26:16,933::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000154995 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) (449) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:16,934::__init__::225::IOProcess::(_processLogs) (449) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:16,935::__init__::225::IOProcess::(_processLogs) (449) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:16,935::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,936::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:16,936::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:26:16,936::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:16,936::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) (450) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) (450) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) (450) Building response >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) (450) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:26:16,937::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) (451) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) (451) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) (451) Building response >Thread-17::DEBUG::2014-08-22 09:26:16,938::__init__::225::IOProcess::(_processLogs) (451) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:16,939::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,939::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:16,939::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:26:16,939::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:16,939::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) (452) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) (452) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) (452) Building response >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) (452) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:16,940::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:16,941::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:26:17,155::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:26:17,155::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cf0c20b-647d-45ee-ab35-1ba7098421e6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:17,156::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:26:17,156::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:17,156::task::1191::Storage.TaskManager.Task::(prepare) Task=`1cf0c20b-647d-45ee-ab35-1ba7098421e6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:17,156::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cf0c20b-647d-45ee-ab35-1ba7098421e6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:17,156::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:17,156::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:17,156::task::993::Storage.TaskManager.Task::(_decref) Task=`1cf0c20b-647d-45ee-ab35-1ba7098421e6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:26:17,181::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:26:17,181::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4bdd531-b8e2-4969-8065-f48e95aacaad`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:26:17,181::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:26:17,182::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f2aa9755-31ed-40e7-ab32-7ef0e53f089a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:26:17,182::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:26:17,182::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:26:17,182::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f2aa9755-31ed-40e7-ab32-7ef0e53f089a`::Granted request >Thread-46::DEBUG::2014-08-22 09:26:17,182::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a4bdd531-b8e2-4969-8065-f48e95aacaad`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:26:17,183::task::993::Storage.TaskManager.Task::(_decref) Task=`a4bdd531-b8e2-4969-8065-f48e95aacaad`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:26:17,184::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:17,185::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4bdd531-b8e2-4969-8065-f48e95aacaad`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:17,185::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4bdd531-b8e2-4969-8065-f48e95aacaad`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:26:17,185::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:26:17,185::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:26:17,185::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:26:17,185::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:26:17,185::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:26:17,185::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:26:17,185::task::993::Storage.TaskManager.Task::(_decref) Task=`a4bdd531-b8e2-4969-8065-f48e95aacaad`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:26:17,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b2c5a90-f04a-4214-8bfc-7a8040b95a2d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:17,328::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:26:17,328::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000638742', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000154995', 'lastCheck': '0.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:17,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b2c5a90-f04a-4214-8bfc-7a8040b95a2d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000638742', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000154995', 'lastCheck': '0.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:17,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b2c5a90-f04a-4214-8bfc-7a8040b95a2d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:17,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:17,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:17,328::task::993::Storage.TaskManager.Task::(_decref) Task=`7b2c5a90-f04a-4214-8bfc-7a8040b95a2d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:17,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:17,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123565 s, 82.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:19,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:19,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116885 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:21,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:21,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130067 s, 78.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:23,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:23,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116319 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:25,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:25,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117475 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:26,694::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:26,694::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:26,694::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:26,694::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:26,694::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,695::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,695::__init__::225::IOProcess::(_processLogs) (453) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:26,695::__init__::225::IOProcess::(_processLogs) (453) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:26,695::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(453) Building response >Thread-21::DEBUG::2014-08-22 09:26:26,696::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:26,696::__init__::225::IOProcess::(_processLogs) (453) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,702::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,703::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:26:26,703::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000190802 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:26,704::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:26,704::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:26,704::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:26,705::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:26,705::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:26,705::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,705::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,705::__init__::225::IOProcess::(_processLogs) (454) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:26,705::__init__::225::IOProcess::(_processLogs) (454) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:26,706::__init__::225::IOProcess::(_processLogs) (454) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,706::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,707::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:26,940::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:26,940::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:26,941::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:26:26,942::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:26,942::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,948::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,948::__init__::225::IOProcess::(_processLogs) (455) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:26:26,949::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000283772 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:26,949::__init__::225::IOProcess::(_processLogs) (455) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:26,950::__init__::225::IOProcess::(_processLogs) (455) Building response >Thread-17::DEBUG::2014-08-22 09:26:26,950::__init__::225::IOProcess::(_processLogs) (455) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,950::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,950::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) (456) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:26,951::__init__::225::IOProcess::(_processLogs) (456) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:26,952::__init__::225::IOProcess::(_processLogs) (456) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,953::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:26,953::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:26:26,953::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:26,953::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:26,953::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) (457) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) (457) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) (457) Building response >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) (457) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:26:26,954::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) (458) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) (458) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) (458) Building response >Thread-17::DEBUG::2014-08-22 09:26:26,955::__init__::225::IOProcess::(_processLogs) (458) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,956::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,956::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:26,956::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:26:26,956::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:26,956::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:26,956::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) (459) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) (459) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) (459) Building response >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) (459) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:26,957::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:26:27,241::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:26:27,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1107ce3-9b73-46ed-b92e-499eb309e487`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:27,242::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:26:27,242::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:27,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1107ce3-9b73-46ed-b92e-499eb309e487`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:27,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1107ce3-9b73-46ed-b92e-499eb309e487`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:27,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:27,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:27,242::task::993::Storage.TaskManager.Task::(_decref) Task=`e1107ce3-9b73-46ed-b92e-499eb309e487`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:26:27,299::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:26:27,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`c538ccd9-de41-46f0-8f50-6c8fd9ad5319`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:26:27,300::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:26:27,300::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e309f528-f387-4f4e-becc-8f4514f5101d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:26:27,300::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:26:27,300::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:26:27,300::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e309f528-f387-4f4e-becc-8f4514f5101d`::Granted request >Thread-46::DEBUG::2014-08-22 09:26:27,301::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c538ccd9-de41-46f0-8f50-6c8fd9ad5319`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:26:27,301::task::993::Storage.TaskManager.Task::(_decref) Task=`c538ccd9-de41-46f0-8f50-6c8fd9ad5319`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:26:27,303::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:27,303::task::1191::Storage.TaskManager.Task::(prepare) Task=`c538ccd9-de41-46f0-8f50-6c8fd9ad5319`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:27,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`c538ccd9-de41-46f0-8f50-6c8fd9ad5319`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:26:27,303::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:26:27,303::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:26:27,303::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:26:27,303::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:26:27,303::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:26:27,303::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:26:27,304::task::993::Storage.TaskManager.Task::(_decref) Task=`c538ccd9-de41-46f0-8f50-6c8fd9ad5319`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:27,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:27,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121355 s, 84.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:29,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:29,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011785 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:31,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:32,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215026 s, 47.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:26:32,550::task::595::Storage.TaskManager.Task::(_updateState) Task=`f14b877a-e112-4c30-b66d-75b476371a72`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:32,551::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:26:32,552::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000190802', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000283772', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:32,552::task::1191::Storage.TaskManager.Task::(prepare) Task=`f14b877a-e112-4c30-b66d-75b476371a72`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000190802', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000283772', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:32,553::task::595::Storage.TaskManager.Task::(_updateState) Task=`f14b877a-e112-4c30-b66d-75b476371a72`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:32,553::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:32,553::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:32,554::task::993::Storage.TaskManager.Task::(_decref) Task=`f14b877a-e112-4c30-b66d-75b476371a72`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:34,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:34,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011693 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:36,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:36,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115967 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:36,707::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:36,707::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:36,707::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,708::__init__::225::IOProcess::(_processLogs) (460) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:36,708::__init__::225::IOProcess::(_processLogs) (460) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:36,708::__init__::225::IOProcess::(_processLogs) (460) Queuing response >Thread-21::DEBUG::2014-08-22 09:26:36,709::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:36,709::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:36,715::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:26:36,715::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00022064 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:36,716::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:36,716::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:36,716::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,716::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,716::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,716::__init__::225::IOProcess::(_processLogs) (461) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:36,717::__init__::225::IOProcess::(_processLogs) (461) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:36,717::__init__::225::IOProcess::(_processLogs) (461) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:36,718::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:36,718::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:36,957::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:36,957::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:36,957::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,958::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,958::__init__::225::IOProcess::(_processLogs) (462) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:36,958::__init__::225::IOProcess::(_processLogs) (462) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:36,958::__init__::225::IOProcess::(_processLogs) (462) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:36,959::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:26:36,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:36,959::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:26:36,965::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216612 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:36,965::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:36,965::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:36,966::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,966::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,966::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,966::__init__::225::IOProcess::(_processLogs) (463) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:36,966::__init__::225::IOProcess::(_processLogs) (463) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:36,967::__init__::225::IOProcess::(_processLogs) (463) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:36,967::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:36,967::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:36,968::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:26:36,968::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:36,968::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,968::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,968::__init__::225::IOProcess::(_processLogs) (464) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) (464) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) (464) Building response >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) (464) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:26:36,969::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) (465) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) (465) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) (465) Building response >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) (465) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:36,970::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:36,971::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:26:36,971::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:36,971::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:36,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) (466) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) (466) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) (466) Building response >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) (466) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:36,972::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:26:37,355::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:26:37,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`1bee74d5-7456-4042-a5de-ce606cc88647`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:37,355::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:26:37,356::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:37,356::task::1191::Storage.TaskManager.Task::(prepare) Task=`1bee74d5-7456-4042-a5de-ce606cc88647`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:37,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`1bee74d5-7456-4042-a5de-ce606cc88647`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:37,356::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:37,356::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:37,356::task::993::Storage.TaskManager.Task::(_decref) Task=`1bee74d5-7456-4042-a5de-ce606cc88647`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:26:37,397::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:26:37,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3bd9d9f-72a6-4022-978f-b6a6d576764e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:26:37,397::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:26:37,397::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d888de2a-94d8-4bed-86fb-184261e78ecd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:26:37,397::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:26:37,397::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:26:37,397::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d888de2a-94d8-4bed-86fb-184261e78ecd`::Granted request >Thread-46::DEBUG::2014-08-22 09:26:37,398::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e3bd9d9f-72a6-4022-978f-b6a6d576764e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:26:37,398::task::993::Storage.TaskManager.Task::(_decref) Task=`e3bd9d9f-72a6-4022-978f-b6a6d576764e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:26:37,400::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:37,400::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3bd9d9f-72a6-4022-978f-b6a6d576764e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:37,400::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3bd9d9f-72a6-4022-978f-b6a6d576764e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:26:37,400::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:26:37,400::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:26:37,400::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:26:37,400::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:26:37,400::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:26:37,400::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:26:37,400::task::993::Storage.TaskManager.Task::(_decref) Task=`e3bd9d9f-72a6-4022-978f-b6a6d576764e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:38,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:38,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123081 s, 83.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:40,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:40,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114945 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:42,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:42,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117322 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:44,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:44,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117876 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:46,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:46,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01147 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) (467) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:46,719::__init__::225::IOProcess::(_processLogs) (467) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:46,720::__init__::225::IOProcess::(_processLogs) (467) Queuing response >Thread-21::DEBUG::2014-08-22 09:26:46,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:46,721::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,727::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:26:46,727::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000497309 s, 798 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:46,728::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:46,728::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,728::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,728::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,728::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,728::__init__::225::IOProcess::(_processLogs) (468) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:46,729::__init__::225::IOProcess::(_processLogs) (468) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:46,729::__init__::225::IOProcess::(_processLogs) (468) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:46,730::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,730::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:46,972::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:46,972::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,972::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,972::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,973::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,973::__init__::225::IOProcess::(_processLogs) (469) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:46,973::__init__::225::IOProcess::(_processLogs) (469) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:46,973::__init__::225::IOProcess::(_processLogs) (469) Queuing response >Thread-22::DEBUG::2014-08-22 09:26:46,974::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:46,974::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,980::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:26:46,981::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223621 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:46,982::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:46,982::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,982::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:46,982::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,982::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,983::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,983::__init__::225::IOProcess::(_processLogs) (470) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:46,983::__init__::225::IOProcess::(_processLogs) (470) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:46,983::__init__::225::IOProcess::(_processLogs) (470) Building response >Thread-17::DEBUG::2014-08-22 09:26:46,983::__init__::225::IOProcess::(_processLogs) (470) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:46,983::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,984::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:46,984::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:46,984::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:26:46,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,984::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) (471) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) (471) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) (471) Building response >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) (471) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:46,985::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:46,986::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:26:46,986::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,986::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:46,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) (472) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) (472) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) (472) Building response >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) (472) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:26:46,987::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:46,988::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:46,988::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) (473) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) (473) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) (473) Building response >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) (473) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:46,989::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:26:47,476::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:26:47,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`78ed100b-251e-4092-8330-af4b219fdad3`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:47,476::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:26:47,476::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:47,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`78ed100b-251e-4092-8330-af4b219fdad3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:47,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`78ed100b-251e-4092-8330-af4b219fdad3`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:47,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:47,477::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:47,477::task::993::Storage.TaskManager.Task::(_decref) Task=`78ed100b-251e-4092-8330-af4b219fdad3`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:26:47,610::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:26:47,610::task::595::Storage.TaskManager.Task::(_updateState) Task=`89da6ff3-18f8-4bc0-9295-675beaee33f3`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:26:47,611::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:26:47,611::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8a330269-4bce-4a16-a71c-3006b619348b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:26:47,611::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:26:47,611::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:26:47,611::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8a330269-4bce-4a16-a71c-3006b619348b`::Granted request >Thread-46::DEBUG::2014-08-22 09:26:47,611::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`89da6ff3-18f8-4bc0-9295-675beaee33f3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:26:47,611::task::993::Storage.TaskManager.Task::(_decref) Task=`89da6ff3-18f8-4bc0-9295-675beaee33f3`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:26:47,613::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:47,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`89da6ff3-18f8-4bc0-9295-675beaee33f3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:47,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`89da6ff3-18f8-4bc0-9295-675beaee33f3`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:26:47,614::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:26:47,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:26:47,614::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:26:47,614::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:26:47,614::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:26:47,614::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:26:47,614::task::993::Storage.TaskManager.Task::(_decref) Task=`89da6ff3-18f8-4bc0-9295-675beaee33f3`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:26:47,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`faa89979-da82-4140-af12-8d9f474ac1ea`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:47,783::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:26:47,783::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000497309', 'lastCheck': '1.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223621', 'lastCheck': '0.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:47,783::task::1191::Storage.TaskManager.Task::(prepare) Task=`faa89979-da82-4140-af12-8d9f474ac1ea`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000497309', 'lastCheck': '1.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223621', 'lastCheck': '0.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:26:47,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`faa89979-da82-4140-af12-8d9f474ac1ea`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:47,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:47,783::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:47,783::task::993::Storage.TaskManager.Task::(_decref) Task=`faa89979-da82-4140-af12-8d9f474ac1ea`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:48,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:48,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011898 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:50,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:50,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117724 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:52,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:52,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116822 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:54,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:54,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166047 s, 61.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:26:56,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:56,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114207 s, 89.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:56,732::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:56,733::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:56,733::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:56,734::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:56,734::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:56,735::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:56,738::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:56,739::__init__::225::IOProcess::(_processLogs) (474) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:26:56,739::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:26:56,740::__init__::225::IOProcess::(_processLogs) (474) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:56,746::__init__::225::IOProcess::(_processLogs) (474) Building response >Thread-21::DEBUG::2014-08-22 09:26:56,747::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000205782 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:56,747::__init__::225::IOProcess::(_processLogs) (474) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:56,749::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:56,749::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:56,750::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:56,751::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:26:56,751::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:56,753::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:56,754::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:56,755::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:56,755::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:56,756::__init__::225::IOProcess::(_processLogs) (475) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:56,757::__init__::225::IOProcess::(_processLogs) (475) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:56,757::__init__::225::IOProcess::(_processLogs) (475) Building response >Thread-17::DEBUG::2014-08-22 09:26:56,758::__init__::225::IOProcess::(_processLogs) (475) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:56,758::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:56,759::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:56,989::__init__::225::IOProcess::(_processLogs) (476) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:56,990::__init__::225::IOProcess::(_processLogs) (476) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:56,990::__init__::225::IOProcess::(_processLogs) (476) Building response >Thread-17::DEBUG::2014-08-22 09:26:56,990::__init__::225::IOProcess::(_processLogs) (476) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:56,990::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:26:56,991::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:26:56,991::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:26:56,997::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000247101 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:26:56,997::__init__::225::IOProcess::(_processLogs) Message size is 162DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) (477) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) (477) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) (477) Building response >Thread-17::DEBUG::2014-08-22 09:26:56,998::__init__::225::IOProcess::(_processLogs) (477) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:56,999::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:26:57,000::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:26:57,000::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:57,000::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:26:57,000::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) (478) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) (478) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) (478) Building response >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) (478) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:57,001::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) (479) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:57,002::__init__::225::IOProcess::(_processLogs) (479) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:57,003::__init__::225::IOProcess::(_processLogs) (479) Building response >Thread-17::DEBUG::2014-08-22 09:26:57,003::__init__::225::IOProcess::(_processLogs) (479) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:57,003::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:57,003::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:26:57,003::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) (480) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) (480) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:26:57,004::__init__::225::IOProcess::(_processLogs) (480) Queuing response >Thread-17::DEBUG::2014-08-22 09:26:57,005::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:26:57,005::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:26:57,662::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:26:57,663::task::595::Storage.TaskManager.Task::(_updateState) Task=`b951d0c4-afe1-4498-ad23-cbd26395d6e0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:26:57,663::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:26:57,663::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:57,663::task::1191::Storage.TaskManager.Task::(prepare) Task=`b951d0c4-afe1-4498-ad23-cbd26395d6e0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:26:57,663::task::595::Storage.TaskManager.Task::(_updateState) Task=`b951d0c4-afe1-4498-ad23-cbd26395d6e0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:26:57,663::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:26:57,663::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:26:57,664::task::993::Storage.TaskManager.Task::(_decref) Task=`b951d0c4-afe1-4498-ad23-cbd26395d6e0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:26:57,687::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:26:57,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`33551f19-3ea8-4c32-8704-fde2997685e3`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:26:57,688::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:26:57,688::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ddfe239a-292f-47be-ab66-b6caf799b130`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:26:57,688::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:26:57,688::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:26:57,688::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ddfe239a-292f-47be-ab66-b6caf799b130`::Granted request >Thread-46::DEBUG::2014-08-22 09:26:57,689::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`33551f19-3ea8-4c32-8704-fde2997685e3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:26:57,689::task::993::Storage.TaskManager.Task::(_decref) Task=`33551f19-3ea8-4c32-8704-fde2997685e3`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:26:57,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:57,691::task::1191::Storage.TaskManager.Task::(prepare) Task=`33551f19-3ea8-4c32-8704-fde2997685e3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:26:57,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`33551f19-3ea8-4c32-8704-fde2997685e3`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:26:57,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:26:57,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:26:57,691::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:26:57,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:26:57,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:26:57,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:26:57,691::task::993::Storage.TaskManager.Task::(_decref) Task=`33551f19-3ea8-4c32-8704-fde2997685e3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:26:58,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:26:58,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119715 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:00,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:00,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118918 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:02,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:02,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021312 s, 48.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:02,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ae10579-e192-46b8-b54a-1b0522edffbe`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:02,958::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:27:02,958::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205782', 'lastCheck': '6.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000247101', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:02,958::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ae10579-e192-46b8-b54a-1b0522edffbe`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205782', 'lastCheck': '6.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000247101', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:02,959::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ae10579-e192-46b8-b54a-1b0522edffbe`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:02,959::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:02,959::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:02,959::task::993::Storage.TaskManager.Task::(_decref) Task=`7ae10579-e192-46b8-b54a-1b0522edffbe`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:04,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:04,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117555 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:06,755::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:06,755::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:06,756::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:27:06,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:06,757::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:06,763::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:27:06,763::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000203954 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:06,764::__init__::225::IOProcess::(_processLogs) (481) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) (481) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) (481) Building response >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) (481) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:06,765::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:06,766::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:06,766::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:06,766::__init__::225::IOProcess::(_processLogs) (482) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:06,766::__init__::225::IOProcess::(_processLogs) (482) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:06,766::__init__::225::IOProcess::(_processLogs) (482) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:06,767::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:06,767::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:27:07,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:07,007::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:07,007::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:07,007::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:07,007::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:07,008::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:07,008::__init__::225::IOProcess::(_processLogs) (483) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:07,008::__init__::225::IOProcess::(_processLogs) (483) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:07,016::__init__::225::IOProcess::(_processLogs) (483) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:07,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:27:07,021::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:27:07,027::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000153716 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:07,029::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:07,029::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:07,030::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:07,031::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:07,033::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:07,034::__init__::225::IOProcess::(_processLogs) (484) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:07,034::__init__::225::IOProcess::(_processLogs) (484) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:07,035::__init__::225::IOProcess::(_processLogs) (484) Building response >Thread-17::DEBUG::2014-08-22 09:27:07,035::__init__::225::IOProcess::(_processLogs) (484) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:07,059::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-45::DEBUG::2014-08-22 09:27:07,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028451 s, 36.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:07,060::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:07,061::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:27:07,062::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:07,063::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:07,064::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:07,064::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:07,064::__init__::225::IOProcess::(_processLogs) (485) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:07,065::__init__::225::IOProcess::(_processLogs) (485) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:07,065::__init__::225::IOProcess::(_processLogs) (485) Building response >Thread-17::DEBUG::2014-08-22 09:27:07,066::__init__::225::IOProcess::(_processLogs) (485) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:07,066::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:07,066::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:07,067::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:27:07,067::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:07,068::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:07,068::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:07,069::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:07,070::__init__::225::IOProcess::(_processLogs) (486) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:07,070::__init__::225::IOProcess::(_processLogs) (486) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:07,071::__init__::225::IOProcess::(_processLogs) (486) Building response >Thread-17::DEBUG::2014-08-22 09:27:07,071::__init__::225::IOProcess::(_processLogs) (486) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:07,071::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:07,072::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:07,072::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:27:07,073::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:07,073::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:07,074::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:07,074::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:07,075::__init__::225::IOProcess::(_processLogs) (487) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:07,076::__init__::225::IOProcess::(_processLogs) (487) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:07,076::__init__::225::IOProcess::(_processLogs) (487) Building response >Thread-17::DEBUG::2014-08-22 09:27:07,077::__init__::225::IOProcess::(_processLogs) (487) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:07,077::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:07,078::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:27:07,742::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:27:07,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`681c7de4-ce1d-476e-8ad1-48956d6b9956`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:07,743::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:27:07,743::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:07,743::task::1191::Storage.TaskManager.Task::(prepare) Task=`681c7de4-ce1d-476e-8ad1-48956d6b9956`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:07,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`681c7de4-ce1d-476e-8ad1-48956d6b9956`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:07,743::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:07,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:07,744::task::993::Storage.TaskManager.Task::(_decref) Task=`681c7de4-ce1d-476e-8ad1-48956d6b9956`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:27:07,776::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:27:07,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`71835693-aa6c-46ce-9270-c6e1f24f872b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:27:07,776::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:27:07,776::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3b2afa8e-d437-4479-ba74-310411cfa0b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:27:07,776::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:27:07,776::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:27:07,776::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3b2afa8e-d437-4479-ba74-310411cfa0b9`::Granted request >Thread-46::DEBUG::2014-08-22 09:27:07,777::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`71835693-aa6c-46ce-9270-c6e1f24f872b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:27:07,777::task::993::Storage.TaskManager.Task::(_decref) Task=`71835693-aa6c-46ce-9270-c6e1f24f872b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:27:07,779::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:07,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`71835693-aa6c-46ce-9270-c6e1f24f872b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:07,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`71835693-aa6c-46ce-9270-c6e1f24f872b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:27:07,779::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:27:07,779::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:27:07,779::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:27:07,779::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:27:07,779::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:27:07,779::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:27:07,780::task::993::Storage.TaskManager.Task::(_decref) Task=`71835693-aa6c-46ce-9270-c6e1f24f872b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:09,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:09,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117958 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:11,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:11,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115909 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:13,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:13,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116131 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:15,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:15,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115971 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:16,768::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:16,768::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:16,768::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:16,768::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:16,768::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:16,768::__init__::225::IOProcess::(_processLogs) (488) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:16,769::__init__::225::IOProcess::(_processLogs) (488) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:16,769::__init__::225::IOProcess::(_processLogs) (488) Queuing response >Thread-21::DEBUG::2014-08-22 09:27:16,770::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:16,770::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:16,776::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:27:16,777::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000203086 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:16,777::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:16,777::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:16,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:16,778::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:16,778::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:16,778::__init__::225::IOProcess::(_processLogs) (489) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:16,778::__init__::225::IOProcess::(_processLogs) (489) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:16,779::__init__::225::IOProcess::(_processLogs) (489) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:16,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:16,779::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:17,076::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:17,076::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:17,076::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:17,076::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:17,077::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:17,077::__init__::225::IOProcess::(_processLogs) (490) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:17,077::__init__::225::IOProcess::(_processLogs) (490) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:17,077::__init__::225::IOProcess::(_processLogs) (490) Queuing response >Thread-22::DEBUG::2014-08-22 09:27:17,078::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:17,078::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:17,084::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:27:17,084::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000202276 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) (491) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:17,085::__init__::225::IOProcess::(_processLogs) (491) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:17,086::__init__::225::IOProcess::(_processLogs) (491) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) (492) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:17,087::__init__::225::IOProcess::(_processLogs) (492) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:17,088::__init__::225::IOProcess::(_processLogs) (492) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:17,088::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:17,088::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:17,088::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:27:17,088::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) (493) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) (493) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) (493) Building response >Thread-17::DEBUG::2014-08-22 09:27:17,089::__init__::225::IOProcess::(_processLogs) (493) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:17,090::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:17,090::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:17,090::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:27:17,090::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:17,090::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) (494) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) (494) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) (494) Building response >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) (494) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:17,091::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:27:17,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:17,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113603 s, 90.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:17,829::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:27:17,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c043a1b-1e1a-440b-96e4-dd8af66dad48`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:17,830::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:27:17,830::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:17,830::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c043a1b-1e1a-440b-96e4-dd8af66dad48`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:17,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c043a1b-1e1a-440b-96e4-dd8af66dad48`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:17,831::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:17,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:17,831::task::993::Storage.TaskManager.Task::(_decref) Task=`5c043a1b-1e1a-440b-96e4-dd8af66dad48`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:27:17,895::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:27:17,895::task::595::Storage.TaskManager.Task::(_updateState) Task=`f492dbfc-bc19-46c3-8957-6a1653b397f0`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:27:17,895::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:27:17,895::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`dc2d6e50-ec53-4c85-b971-892e5442a018`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:27:17,896::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:27:17,896::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:27:17,896::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`dc2d6e50-ec53-4c85-b971-892e5442a018`::Granted request >Thread-46::DEBUG::2014-08-22 09:27:17,896::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f492dbfc-bc19-46c3-8957-6a1653b397f0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:27:17,896::task::993::Storage.TaskManager.Task::(_decref) Task=`f492dbfc-bc19-46c3-8957-6a1653b397f0`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:27:17,898::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:17,898::task::1191::Storage.TaskManager.Task::(prepare) Task=`f492dbfc-bc19-46c3-8957-6a1653b397f0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:17,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`f492dbfc-bc19-46c3-8957-6a1653b397f0`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:27:17,898::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:27:17,898::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:27:17,899::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:27:17,899::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:27:17,899::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:27:17,899::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:27:17,899::task::993::Storage.TaskManager.Task::(_decref) Task=`f492dbfc-bc19-46c3-8957-6a1653b397f0`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:27:18,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`2420d7a5-4aa5-4ec9-8887-548cdc9b7b01`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:18,168::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:27:18,169::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000203086', 'lastCheck': '1.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000202276', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:18,169::task::1191::Storage.TaskManager.Task::(prepare) Task=`2420d7a5-4aa5-4ec9-8887-548cdc9b7b01`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000203086', 'lastCheck': '1.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000202276', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:18,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`2420d7a5-4aa5-4ec9-8887-548cdc9b7b01`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:18,169::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:18,169::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:18,169::task::993::Storage.TaskManager.Task::(_decref) Task=`2420d7a5-4aa5-4ec9-8887-548cdc9b7b01`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:19,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:19,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118431 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:21,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:21,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117524 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:23,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:23,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117147 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:25,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:25,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117391 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:26,780::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:26,780::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:26,780::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:26,780::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:26,780::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:26,780::__init__::225::IOProcess::(_processLogs) (495) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:26,781::__init__::225::IOProcess::(_processLogs) (495) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:26,781::__init__::225::IOProcess::(_processLogs) (495) Queuing response >Thread-21::DEBUG::2014-08-22 09:27:26,782::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:26,782::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:26,788::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:27:26,789::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000193803 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:26,790::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:26,790::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:26,790::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:26,790::__init__::225::IOProcess::(_processLogs) Message size is 160DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:26,791::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:26,791::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:26,791::__init__::225::IOProcess::(_processLogs) (496) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:26,792::__init__::225::IOProcess::(_processLogs) (496) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:26,792::__init__::225::IOProcess::(_processLogs) (496) Building response >Thread-17::DEBUG::2014-08-22 09:27:26,792::__init__::225::IOProcess::(_processLogs) (496) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:26,792::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:26,792::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:27,091::__init__::225::IOProcess::(_processLogs) Message size is 160DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:27,091::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:27,091::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) Message size is 160DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) (497) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) (497) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) (497) Building response >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) (497) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:27,092::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:27,093::__init__::225::IOProcess::(_processLogs) Message size is 160DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:27:27,093::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:27:27,099::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00023175 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Message size is 160DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:27,100::__init__::225::IOProcess::(_processLogs) (498) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:27,101::__init__::225::IOProcess::(_processLogs) (498) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:27,101::__init__::225::IOProcess::(_processLogs) (498) Building response >Thread-17::DEBUG::2014-08-22 09:27:27,101::__init__::225::IOProcess::(_processLogs) (498) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:27,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:27,102::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:27:27,102::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:27,102::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:27,102::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) (499) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) (499) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) (499) Building response >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) (499) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:27:27,103::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:27,104::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:27,104::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:27,104::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:27,104::__init__::225::IOProcess::(_processLogs) (500) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:27,104::__init__::225::IOProcess::(_processLogs) (500) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) (500) Building response >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) (500) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:27,105::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) (501) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) (501) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) (501) Building response >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) (501) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:27,106::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:27,107::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:27:27,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:27,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119093 s, 86.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:28,033::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:27:28,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5cdc507-f4e5-43b2-896d-783bc6c6e8b3`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:28,033::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:27:28,034::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:28,034::task::1191::Storage.TaskManager.Task::(prepare) Task=`e5cdc507-f4e5-43b2-896d-783bc6c6e8b3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:28,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5cdc507-f4e5-43b2-896d-783bc6c6e8b3`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:28,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:28,034::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:28,034::task::993::Storage.TaskManager.Task::(_decref) Task=`e5cdc507-f4e5-43b2-896d-783bc6c6e8b3`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:27:28,087::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:27:28,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a6952f3-7776-48f7-96dc-f635fa65276b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:27:28,087::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:27:28,087::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7fbc218f-8867-4af4-9c38-4296b3135fc2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:27:28,087::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:27:28,088::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:27:28,088::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7fbc218f-8867-4af4-9c38-4296b3135fc2`::Granted request >Thread-46::DEBUG::2014-08-22 09:27:28,088::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0a6952f3-7776-48f7-96dc-f635fa65276b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:27:28,088::task::993::Storage.TaskManager.Task::(_decref) Task=`0a6952f3-7776-48f7-96dc-f635fa65276b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:27:28,090::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:28,090::task::1191::Storage.TaskManager.Task::(prepare) Task=`0a6952f3-7776-48f7-96dc-f635fa65276b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:28,090::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a6952f3-7776-48f7-96dc-f635fa65276b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:27:28,090::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:27:28,090::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:27:28,090::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:27:28,090::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:27:28,091::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:27:28,091::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:27:28,091::task::993::Storage.TaskManager.Task::(_decref) Task=`0a6952f3-7776-48f7-96dc-f635fa65276b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:29,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:29,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011687 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:31,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:31,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011599 s, 88.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:33,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`05fb1fc8-2098-43a3-b8e9-7923eb273636`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:33,352::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:27:33,352::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193803', 'lastCheck': '6.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00023175', 'lastCheck': '6.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:33,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`05fb1fc8-2098-43a3-b8e9-7923eb273636`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193803', 'lastCheck': '6.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00023175', 'lastCheck': '6.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:33,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`05fb1fc8-2098-43a3-b8e9-7923eb273636`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:33,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:33,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:33,352::task::993::Storage.TaskManager.Task::(_decref) Task=`05fb1fc8-2098-43a3-b8e9-7923eb273636`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:33,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:33,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115857 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:35,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:35,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116865 s, 87.6 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:27:36,791::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:27:36,793::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:27:36,793::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,793::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,793::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,794::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,794::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,794::__init__::225::IOProcess::(_processLogs) (502) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:27:36,794::__init__::225::IOProcess::(_processLogs) (502) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:27:36,794::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(502) Building response >Thread-17::DEBUG::2014-08-22 09:27:36,795::__init__::225::IOProcess::(_processLogs) (502) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,796::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:36,797::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:27:36,798::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,799::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:27:36,800::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,801::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,802::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,804::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,805::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,806::__init__::225::IOProcess::(_processLogs) (503) Finding callback 'glob'... >Thread-21::DEBUG::2014-08-22 09:27:36,806::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:27:36,807::__init__::225::IOProcess::(_processLogs) (503) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:27:36,807::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(503) Building response >Thread-17::DEBUG::2014-08-22 09:27:36,807::__init__::225::IOProcess::(_processLogs) (503) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Message size is 169 >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,808::__init__::225::IOProcess::(_processLogs) (504) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:27:36,809::__init__::225::IOProcess::(_processLogs) (504) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:27:36,829::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(504) Building response >Thread-17::DEBUG::2014-08-22 09:27:36,829::__init__::225::IOProcess::(_processLogs) (504) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,829::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:27:36,830::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:27:36,830::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Message size is 193 >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,831::__init__::225::IOProcess::(_processLogs) (505) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:27:36,832::__init__::225::IOProcess::(_processLogs) (505) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:27:36,832::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(505) Building response >Thread-21::DEBUG::2014-08-22 09:27:36,832::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:27:36,833::__init__::225::IOProcess::(_processLogs) (505) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,834::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:36,834::__init__::225::IOProcess::(_processLogs) Sending response sized 603 >Thread-17::DEBUG::2014-08-22 09:27:36,834::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,834::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:27:36,834::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) (506) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) (506) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) (506) Building response >Thread-17::DEBUG::2014-08-22 09:27:36,835::__init__::225::IOProcess::(_processLogs) (506) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,836::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,837::__init__::225::IOProcess::(_processLogs) (507) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:27:36,837::__init__::225::IOProcess::(_processLogs) (507) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:27:36,837::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(507) Building response >Thread-21::DEBUG::2014-08-22 09:27:36,837::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:27:36,838::__init__::225::IOProcess::(_processLogs) (507) Queuing response >Thread-21::INFO::2014-08-22 09:27:36,838::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:27:36,838::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 09:27:36,838::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:27:36,839::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-17::DEBUG::2014-08-22 09:27:36,839::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,839::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:36,839::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,840::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:27:36,841::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:36,841::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,847::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:27:36,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000195504 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:36,848::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,849::__init__::225::IOProcess::(_processLogs) (508) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:36,849::__init__::225::IOProcess::(_processLogs) (508) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:36,849::__init__::225::IOProcess::(_processLogs) (508) Building response >Thread-17::DEBUG::2014-08-22 09:27:36,849::__init__::225::IOProcess::(_processLogs) (508) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,849::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:36,849::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) (509) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:36,850::__init__::225::IOProcess::(_processLogs) (509) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:36,851::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(509) Building response >Thread-17::DEBUG::2014-08-22 09:27:36,852::__init__::225::IOProcess::(_processLogs) (509) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:36,852::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:36,852::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:27:37,106::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:27:37,107::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,108::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:27:37,108::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,108::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) (510) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) (510) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) (510) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) (510) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:27:37,109::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,110::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:27:37,110::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,110::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,110::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,111::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,112::__init__::225::IOProcess::(_processLogs) (511) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:27:37,112::__init__::225::IOProcess::(_processLogs) (511) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:27:37,113::__init__::225::IOProcess::(_processLogs) (511) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,114::__init__::225::IOProcess::(_processLogs) (511) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,114::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,115::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-22::DEBUG::2014-08-22 09:27:37,116::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:27:37,116::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,116::__init__::225::IOProcess::(_processLogs) Message size is 171 >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) (512) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:27:37,117::__init__::225::IOProcess::(_processLogs) (512) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:27:37,138::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(512) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,139::__init__::225::IOProcess::(_processLogs) (512) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,139::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:27:37,139::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:27:37,140::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:37,140::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,140::__init__::225::IOProcess::(_processLogs) Message size is 195 >Thread-17::DEBUG::2014-08-22 09:27:37,140::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,140::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,140::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,141::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,141::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,141::__init__::225::IOProcess::(_processLogs) (513) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:27:37,141::__init__::225::IOProcess::(_processLogs) (513) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:27:37,141::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(513) Building response >Thread-22::DEBUG::2014-08-22 09:27:37,141::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:27:37,142::__init__::225::IOProcess::(_processLogs) (513) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,142::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,143::__init__::225::IOProcess::(_processLogs) Sending response sized 831 >Thread-17::DEBUG::2014-08-22 09:27:37,143::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,143::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-17::DEBUG::2014-08-22 09:27:37,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,143::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) (514) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) (514) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) (514) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) (514) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:37,144::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) (515) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) (515) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) (515) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,145::__init__::225::IOProcess::(_processLogs) (515) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,146::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:27:37,146::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-22::INFO::2014-08-22 09:27:37,146::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:27:37,146::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-22::INFO::2014-08-22 09:27:37,146::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:27:37,147::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,147::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:37,147::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,148::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,148::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:27:37,148::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:37,149::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,155::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:27:37,155::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000192995 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:37,156::__init__::225::IOProcess::(_processLogs) (516) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:37,156::__init__::225::IOProcess::(_processLogs) (516) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:37,156::__init__::225::IOProcess::(_processLogs) (516) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) (516) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,157::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,158::__init__::225::IOProcess::(_processLogs) (517) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:37,158::__init__::225::IOProcess::(_processLogs) (517) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:37,158::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(517) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,159::__init__::225::IOProcess::(_processLogs) (517) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,159::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,159::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:37,159::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,159::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:27:37,159::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,160::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,160::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,160::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,160::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,160::__init__::225::IOProcess::(_processLogs) (518) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:37,160::__init__::225::IOProcess::(_processLogs) (518) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) (518) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) (518) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:27:37,161::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) (519) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) (519) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) (519) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) (519) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,162::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,163::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:37,163::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:37,163::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:27:37,163::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:37,163::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:37,163::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:37,164::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:37,164::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:37,164::__init__::225::IOProcess::(_processLogs) (520) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:37,164::__init__::225::IOProcess::(_processLogs) (520) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:37,164::__init__::225::IOProcess::(_processLogs) (520) Building response >Thread-17::DEBUG::2014-08-22 09:27:37,164::__init__::225::IOProcess::(_processLogs) (520) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:37,165::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:37,165::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:27:37,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:37,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116011 s, 88.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:38,153::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:27:38,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9b712ad-753c-4218-8e8c-f203cd732106`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:38,153::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:27:38,154::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:38,154::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9b712ad-753c-4218-8e8c-f203cd732106`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:38,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9b712ad-753c-4218-8e8c-f203cd732106`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:38,154::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:38,154::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:38,154::task::993::Storage.TaskManager.Task::(_decref) Task=`b9b712ad-753c-4218-8e8c-f203cd732106`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:27:38,192::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:27:38,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`63c8b849-6043-4efe-9526-c7e37577017d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:27:38,193::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:27:38,193::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`248d2ccd-fdae-4262-9b97-311052d51558`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:27:38,193::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:27:38,193::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:27:38,193::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`248d2ccd-fdae-4262-9b97-311052d51558`::Granted request >Thread-46::DEBUG::2014-08-22 09:27:38,194::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`63c8b849-6043-4efe-9526-c7e37577017d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:27:38,194::task::993::Storage.TaskManager.Task::(_decref) Task=`63c8b849-6043-4efe-9526-c7e37577017d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:27:38,196::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:38,196::task::1191::Storage.TaskManager.Task::(prepare) Task=`63c8b849-6043-4efe-9526-c7e37577017d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:38,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`63c8b849-6043-4efe-9526-c7e37577017d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:27:38,196::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:27:38,196::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:27:38,196::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:27:38,196::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:27:38,196::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:27:38,196::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:27:38,196::task::993::Storage.TaskManager.Task::(_decref) Task=`63c8b849-6043-4efe-9526-c7e37577017d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:39,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:39,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117139 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:41,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:41,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116594 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:43,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:43,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116422 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:45,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:46,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116478 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:46,852::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:46,852::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) (521) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:46,853::__init__::225::IOProcess::(_processLogs) (521) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:46,854::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(521) Building response >Thread-21::DEBUG::2014-08-22 09:27:46,854::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:46,855::__init__::225::IOProcess::(_processLogs) (521) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:46,860::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:46,860::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:27:46,861::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000186726 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:46,861::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:46,861::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:46,861::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:46,861::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:46,862::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:46,862::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:46,862::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:46,862::__init__::225::IOProcess::(_processLogs) (522) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:46,862::__init__::225::IOProcess::(_processLogs) (522) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:46,863::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(522) Building response >Thread-17::DEBUG::2014-08-22 09:27:46,863::__init__::225::IOProcess::(_processLogs) (522) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:46,863::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:46,864::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:47,164::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:47,164::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:47,165::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:47,165::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:27:47,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:47,166::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:47,171::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:47,172::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:47,172::__init__::225::IOProcess::(_processLogs) (523) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:47,172::__init__::225::IOProcess::(_processLogs) (523) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:47,172::__init__::225::IOProcess::(_processLogs) (523) Building response >Thread-17::DEBUG::2014-08-22 09:27:47,172::__init__::225::IOProcess::(_processLogs) (523) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:47,172::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:47,173::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:27:47,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198402 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:47,173::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) (524) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) (524) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) (524) Building response >Thread-17::DEBUG::2014-08-22 09:27:47,174::__init__::225::IOProcess::(_processLogs) (524) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:47,175::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:27:47,176::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:47,176::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:47,176::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:27:47,176::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:47,176::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) (525) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) (525) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) (525) Building response >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) (525) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:47,177::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:47,178::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:27:47,178::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:47,178::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:47,178::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:47,178::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:47,178::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) (526) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) (526) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) (526) Building response >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) (526) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:27:47,179::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:47,180::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:47,180::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:47,180::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:47,180::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:47,180::__init__::225::IOProcess::(_processLogs) (527) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:47,181::__init__::225::IOProcess::(_processLogs) (527) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:47,181::__init__::225::IOProcess::(_processLogs) (527) Building response >Thread-17::DEBUG::2014-08-22 09:27:47,181::__init__::225::IOProcess::(_processLogs) (527) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:47,181::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:47,181::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:27:48,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:48,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119213 s, 85.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:48,245::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:27:48,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f7963fa-374c-4bbb-8656-51c208d1a010`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:48,246::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:27:48,246::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:48,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f7963fa-374c-4bbb-8656-51c208d1a010`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:48,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f7963fa-374c-4bbb-8656-51c208d1a010`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:48,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:48,246::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:48,246::task::993::Storage.TaskManager.Task::(_decref) Task=`4f7963fa-374c-4bbb-8656-51c208d1a010`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:27:48,301::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:27:48,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab0f77e-3088-49ad-827a-a1a53ca6bd60`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:27:48,301::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:27:48,302::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`14f4f6de-5a04-4c81-ac1a-daeafa8cd515`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:27:48,302::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:27:48,302::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:27:48,302::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`14f4f6de-5a04-4c81-ac1a-daeafa8cd515`::Granted request >Thread-46::DEBUG::2014-08-22 09:27:48,302::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bab0f77e-3088-49ad-827a-a1a53ca6bd60`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:27:48,302::task::993::Storage.TaskManager.Task::(_decref) Task=`bab0f77e-3088-49ad-827a-a1a53ca6bd60`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:27:48,304::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:48,304::task::1191::Storage.TaskManager.Task::(prepare) Task=`bab0f77e-3088-49ad-827a-a1a53ca6bd60`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:48,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab0f77e-3088-49ad-827a-a1a53ca6bd60`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:27:48,304::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:27:48,305::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:27:48,305::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:27:48,305::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:27:48,305::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:27:48,305::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:27:48,305::task::993::Storage.TaskManager.Task::(_decref) Task=`bab0f77e-3088-49ad-827a-a1a53ca6bd60`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:27:48,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3119516-6214-4018-9519-813c10fe7ddf`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:48,731::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:27:48,731::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000186726', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198402', 'lastCheck': '1.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:48,731::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3119516-6214-4018-9519-813c10fe7ddf`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000186726', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198402', 'lastCheck': '1.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:27:48,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3119516-6214-4018-9519-813c10fe7ddf`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:48,731::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:48,731::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:48,731::task::993::Storage.TaskManager.Task::(_decref) Task=`e3119516-6214-4018-9519-813c10fe7ddf`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:27:50,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:50,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117483 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:52,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:52,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011709 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:54,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:54,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114257 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:27:56,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:56,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116239 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:56,864::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:56,864::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:56,865::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-21::DEBUG::2014-08-22 09:27:56,865::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:56,866::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:56,871::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:56,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:27:56,873::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187641 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:56,873::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:56,874::__init__::225::IOProcess::(_processLogs) (528) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:56,874::__init__::225::IOProcess::(_processLogs) (528) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:56,874::__init__::225::IOProcess::(_processLogs) (528) Building response >Thread-17::DEBUG::2014-08-22 09:27:56,874::__init__::225::IOProcess::(_processLogs) (528) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:56,874::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:56,874::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) (529) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:56,875::__init__::225::IOProcess::(_processLogs) (529) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:56,876::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(529) Building response >Thread-17::DEBUG::2014-08-22 09:27:56,877::__init__::225::IOProcess::(_processLogs) (529) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:56,877::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:56,877::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) (530) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:57,181::__init__::225::IOProcess::(_processLogs) (530) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:57,182::__init__::225::IOProcess::(_processLogs) (530) Building response >Thread-17::DEBUG::2014-08-22 09:27:57,182::__init__::225::IOProcess::(_processLogs) (530) Queuing response >Thread-22::DEBUG::2014-08-22 09:27:57,183::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:27:57,183::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:57,189::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:27:57,190::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000221748 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:27:57,190::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:27:57,190::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:27:57,190::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:57,190::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:57,190::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:57,191::__init__::225::IOProcess::(_processLogs) (531) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:27:57,191::__init__::225::IOProcess::(_processLogs) (531) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:27:57,191::__init__::225::IOProcess::(_processLogs) (531) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:57,192::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:57,192::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:27:57,192::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:27:57,192::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) (532) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) (532) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) (532) Building response >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) (532) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:57,193::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) (533) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:57,194::__init__::225::IOProcess::(_processLogs) (533) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:57,195::__init__::225::IOProcess::(_processLogs) (533) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:57,195::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:57,195::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:27:57,195::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:27:57,195::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:27:57,195::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:27:57,196::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:27:57,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:27:57,196::__init__::225::IOProcess::(_processLogs) (534) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:27:57,196::__init__::225::IOProcess::(_processLogs) (534) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:27:57,196::__init__::225::IOProcess::(_processLogs) (534) Queuing response >Thread-17::DEBUG::2014-08-22 09:27:57,196::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:27:57,197::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:27:58,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:27:58,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116143 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:27:58,381::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:27:58,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`72e50e28-7c07-491a-a23b-dee5c0fbb5bb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:27:58,381::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:27:58,382::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:58,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`72e50e28-7c07-491a-a23b-dee5c0fbb5bb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:27:58,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`72e50e28-7c07-491a-a23b-dee5c0fbb5bb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:27:58,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:27:58,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:27:58,382::task::993::Storage.TaskManager.Task::(_decref) Task=`72e50e28-7c07-491a-a23b-dee5c0fbb5bb`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:27:58,425::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:27:58,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`392fe4c5-7558-4697-84ea-51da5aece1cc`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:27:58,425::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:27:58,425::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3ad3bdc7-0986-4786-8557-6cfa39c0aa1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:27:58,425::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:27:58,425::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:27:58,426::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3ad3bdc7-0986-4786-8557-6cfa39c0aa1f`::Granted request >Thread-46::DEBUG::2014-08-22 09:27:58,426::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`392fe4c5-7558-4697-84ea-51da5aece1cc`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:27:58,426::task::993::Storage.TaskManager.Task::(_decref) Task=`392fe4c5-7558-4697-84ea-51da5aece1cc`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:27:58,428::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:58,428::task::1191::Storage.TaskManager.Task::(prepare) Task=`392fe4c5-7558-4697-84ea-51da5aece1cc`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:27:58,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`392fe4c5-7558-4697-84ea-51da5aece1cc`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:27:58,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:27:58,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:27:58,428::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:27:58,428::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:27:58,429::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:27:58,429::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:27:58,429::task::993::Storage.TaskManager.Task::(_decref) Task=`392fe4c5-7558-4697-84ea-51da5aece1cc`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:00,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:00,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115852 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:02,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:02,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211828 s, 48.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:28:04,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`761db6dd-ec75-47f9-9296-58f48addf33d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:04,026::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:28:04,026::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187641', 'lastCheck': '7.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221748', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:04,026::task::1191::Storage.TaskManager.Task::(prepare) Task=`761db6dd-ec75-47f9-9296-58f48addf33d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187641', 'lastCheck': '7.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221748', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:04,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`761db6dd-ec75-47f9-9296-58f48addf33d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:04,026::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:04,026::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:04,026::task::993::Storage.TaskManager.Task::(_decref) Task=`761db6dd-ec75-47f9-9296-58f48addf33d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:04,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:04,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114758 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:06,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:06,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118712 s, 86.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:06,877::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:06,877::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:06,878::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:06,878::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:06,878::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:06,878::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:06,878::__init__::225::IOProcess::(_processLogs) (535) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:06,878::__init__::225::IOProcess::(_processLogs) (535) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:06,879::__init__::225::IOProcess::(_processLogs) DEBUG|(535) Building response >Thread-21::DEBUG::2014-08-22 09:28:06,879::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:06,879::__init__::225::IOProcess::(_processLogs) (535) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:06,885::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:28:06,886::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196946 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:06,886::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) (536) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:06,888::__init__::225::IOProcess::(_processLogs) (536) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:06,889::__init__::225::IOProcess::(_processLogs) DEBUG|(536) Building response >Thread-17::DEBUG::2014-08-22 09:28:06,890::__init__::225::IOProcess::(_processLogs) (536) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:06,890::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:06,890::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:07,197::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:07,197::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:07,197::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:07,198::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:28:07,199::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:07,199::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:07,204::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:07,204::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:07,204::__init__::225::IOProcess::(_processLogs) (537) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:07,204::__init__::225::IOProcess::(_processLogs) (537) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:07,205::__init__::225::IOProcess::(_processLogs) (537) Building response >Thread-17::DEBUG::2014-08-22 09:28:07,205::__init__::225::IOProcess::(_processLogs) (537) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:07,205::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:07,205::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:28:07,205::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000185181 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:07,206::__init__::225::IOProcess::(_processLogs) (538) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:07,207::__init__::225::IOProcess::(_processLogs) (538) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:07,207::__init__::225::IOProcess::(_processLogs) (538) Building response >Thread-17::DEBUG::2014-08-22 09:28:07,207::__init__::225::IOProcess::(_processLogs) (538) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:07,207::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:07,207::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:07,208::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:07,208::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:28:07,208::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) (539) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) (539) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:07,209::__init__::225::IOProcess::(_processLogs) (539) Building response >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) (539) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:07,210::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) (540) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) (540) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) (540) Building response >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) (540) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:07,211::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:07,212::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:07,212::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:28:07,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:07,212::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:07,212::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) (541) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) (541) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) (541) Building response >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) (541) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:07,213::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:28:08,489::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:28:08,489::task::595::Storage.TaskManager.Task::(_updateState) Task=`379946bb-a262-4045-bf56-8555cca2f09e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:08,489::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:28:08,489::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:08,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`379946bb-a262-4045-bf56-8555cca2f09e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:08,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`379946bb-a262-4045-bf56-8555cca2f09e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:08,490::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:08,490::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:08,490::task::993::Storage.TaskManager.Task::(_decref) Task=`379946bb-a262-4045-bf56-8555cca2f09e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:28:08,569::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:28:08,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`599ff835-60be-47bb-b146-a626d695a3c7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:28:08,569::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:28:08,570::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7ab09e78-79bf-42ec-afe0-b1f3bb55d1cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:28:08,570::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:28:08,570::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:28:08,570::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7ab09e78-79bf-42ec-afe0-b1f3bb55d1cb`::Granted request >Thread-46::DEBUG::2014-08-22 09:28:08,570::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`599ff835-60be-47bb-b146-a626d695a3c7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:28:08,570::task::993::Storage.TaskManager.Task::(_decref) Task=`599ff835-60be-47bb-b146-a626d695a3c7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:28:08,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:08,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`599ff835-60be-47bb-b146-a626d695a3c7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:08,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`599ff835-60be-47bb-b146-a626d695a3c7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:28:08,573::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:28:08,573::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:28:08,573::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:28:08,573::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:28:08,573::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:28:08,573::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:28:08,573::task::993::Storage.TaskManager.Task::(_decref) Task=`599ff835-60be-47bb-b146-a626d695a3c7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:08,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:08,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116949 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:10,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:10,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116144 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:12,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:12,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115686 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:14,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:14,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166595 s, 61.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:16,890::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) (542) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:16,891::__init__::225::IOProcess::(_processLogs) (542) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:16,892::__init__::225::IOProcess::(_processLogs) DEBUG|(542) Building response >Thread-21::DEBUG::2014-08-22 09:28:16,893::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:16,893::__init__::225::IOProcess::(_processLogs) (542) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:16,898::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:16,899::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:28:16,899::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189681 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) (543) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:16,900::__init__::225::IOProcess::(_processLogs) (543) Got request for method 'statvfs' >Dummy-45::DEBUG::2014-08-22 09:28:16,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:16,908::__init__::225::IOProcess::(_processLogs) DEBUG|(543) Building response >Thread-17::DEBUG::2014-08-22 09:28:16,909::__init__::225::IOProcess::(_processLogs) (543) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:16,909::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:16,909::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:28:16,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138819 s, 73.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:17,213::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:17,213::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:17,213::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) (544) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) (544) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:17,214::__init__::225::IOProcess::(_processLogs) DEBUG|(544) Building response >Thread-17::DEBUG::2014-08-22 09:28:17,215::__init__::225::IOProcess::(_processLogs) (544) Queuing response >Thread-22::DEBUG::2014-08-22 09:28:17,215::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:17,215::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:17,221::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:28:17,221::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00024465 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:17,222::__init__::225::IOProcess::(_processLogs) (545) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:17,223::__init__::225::IOProcess::(_processLogs) (545) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:17,223::__init__::225::IOProcess::(_processLogs) (545) Building response >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) (545) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:17,224::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:17,225::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:17,225::__init__::225::IOProcess::(_processLogs) (546) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:17,225::__init__::225::IOProcess::(_processLogs) (546) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:17,225::__init__::225::IOProcess::(_processLogs) (546) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:17,225::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:17,226::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:17,226::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:28:17,226::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:17,226::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:17,226::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) (547) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) (547) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) (547) Building response >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) (547) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:28:17,227::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:17,228::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:17,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:17,228::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:17,229::__init__::225::IOProcess::(_processLogs) (548) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:17,229::__init__::225::IOProcess::(_processLogs) (548) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:17,229::__init__::225::IOProcess::(_processLogs) (548) Building response >Thread-17::DEBUG::2014-08-22 09:28:17,229::__init__::225::IOProcess::(_processLogs) (548) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:17,229::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:17,229::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:28:18,621::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:28:18,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ba6e13c-a2c6-4e6b-8c3c-26ce40843619`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:18,622::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:28:18,622::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:18,622::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ba6e13c-a2c6-4e6b-8c3c-26ce40843619`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:18,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ba6e13c-a2c6-4e6b-8c3c-26ce40843619`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:18,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:18,623::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:18,623::task::993::Storage.TaskManager.Task::(_decref) Task=`7ba6e13c-a2c6-4e6b-8c3c-26ce40843619`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:28:18,665::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:28:18,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ec40839-ff16-4d30-8cd4-ea58863a5de9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:28:18,665::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:28:18,665::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8d97c68d-b8b3-4a62-b50b-94f34a7d250e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:28:18,665::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:28:18,666::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:28:18,666::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8d97c68d-b8b3-4a62-b50b-94f34a7d250e`::Granted request >Thread-46::DEBUG::2014-08-22 09:28:18,666::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3ec40839-ff16-4d30-8cd4-ea58863a5de9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:28:18,666::task::993::Storage.TaskManager.Task::(_decref) Task=`3ec40839-ff16-4d30-8cd4-ea58863a5de9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:28:18,668::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:18,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ec40839-ff16-4d30-8cd4-ea58863a5de9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:18,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ec40839-ff16-4d30-8cd4-ea58863a5de9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:28:18,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:28:18,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:28:18,668::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:28:18,668::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:28:18,668::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:28:18,669::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:28:18,669::task::993::Storage.TaskManager.Task::(_decref) Task=`3ec40839-ff16-4d30-8cd4-ea58863a5de9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:18,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:18,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137664 s, 74.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:28:19,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`f32ae23b-12d7-42e9-a3ad-a70251469793`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:19,226::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:28:19,226::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189681', 'lastCheck': '2.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00024465', 'lastCheck': '2.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:19,226::task::1191::Storage.TaskManager.Task::(prepare) Task=`f32ae23b-12d7-42e9-a3ad-a70251469793`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189681', 'lastCheck': '2.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00024465', 'lastCheck': '2.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:19,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`f32ae23b-12d7-42e9-a3ad-a70251469793`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:19,226::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:19,226::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:19,226::task::993::Storage.TaskManager.Task::(_decref) Task=`f32ae23b-12d7-42e9-a3ad-a70251469793`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:21,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:21,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116486 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:23,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:23,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011501 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:25,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:25,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119832 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:26,909::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:26,910::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-21::DEBUG::2014-08-22 09:28:26,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:26,911::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:26,917::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:28:26,918::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000215725 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:26,918::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:26,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:26,919::__init__::225::IOProcess::(_processLogs) (549) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:26,919::__init__::225::IOProcess::(_processLogs) (549) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:26,919::__init__::225::IOProcess::(_processLogs) (549) Building response >Thread-17::DEBUG::2014-08-22 09:28:26,919::__init__::225::IOProcess::(_processLogs) (549) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:26,920::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:26,921::__init__::225::IOProcess::(_processLogs) (550) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:26,921::__init__::225::IOProcess::(_processLogs) (550) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:26,921::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(550) Building response >Thread-17::DEBUG::2014-08-22 09:28:26,922::__init__::225::IOProcess::(_processLogs) (550) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:26,922::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:26,922::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:28:27,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:27,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121799 s, 84.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:27,229::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:27,230::__init__::225::IOProcess::(_processLogs) (551) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:27,230::__init__::225::IOProcess::(_processLogs) (551) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:27,230::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(551) Building response >Thread-22::DEBUG::2014-08-22 09:28:27,231::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:27,231::__init__::225::IOProcess::(_processLogs) (551) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:27,237::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:28:27,238::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000185931 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:27,238::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:27,239::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:27,239::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) (552) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:27,240::__init__::225::IOProcess::(_processLogs) (552) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:27,241::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(552) Building response >Thread-17::DEBUG::2014-08-22 09:28:27,241::__init__::225::IOProcess::(_processLogs) (552) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:27,241::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:27,242::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:27,242::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:27,242::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:28:27,242::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:27,242::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) (553) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) (553) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) (553) Building response >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) (553) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:27,243::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:27,244::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:28:27,244::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:27,244::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:27,244::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:27,244::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:27,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) (554) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) (554) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) (554) Building response >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) (554) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:28:27,245::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:27,246::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:27,246::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:27,246::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:27,246::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:27,247::__init__::225::IOProcess::(_processLogs) (555) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:27,247::__init__::225::IOProcess::(_processLogs) (555) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:27,247::__init__::225::IOProcess::(_processLogs) (555) Building response >Thread-17::DEBUG::2014-08-22 09:28:27,247::__init__::225::IOProcess::(_processLogs) (555) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:27,247::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:27,247::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:28:28,754::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:28:28,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`1002e379-0537-4527-9a83-b364cd4bacde`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:28,754::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:28:28,754::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:28,754::task::1191::Storage.TaskManager.Task::(prepare) Task=`1002e379-0537-4527-9a83-b364cd4bacde`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:28,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`1002e379-0537-4527-9a83-b364cd4bacde`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:28,755::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:28,755::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:28,755::task::993::Storage.TaskManager.Task::(_decref) Task=`1002e379-0537-4527-9a83-b364cd4bacde`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:28:28,878::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:28:28,878::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ddb22de-b2e5-4642-b700-d1ca500bf43b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:28:28,879::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:28:28,879::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`01508708-f72a-4e92-85f2-d7c32524f23a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:28:28,879::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:28:28,879::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:28:28,879::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`01508708-f72a-4e92-85f2-d7c32524f23a`::Granted request >Thread-46::DEBUG::2014-08-22 09:28:28,879::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6ddb22de-b2e5-4642-b700-d1ca500bf43b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:28:28,879::task::993::Storage.TaskManager.Task::(_decref) Task=`6ddb22de-b2e5-4642-b700-d1ca500bf43b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:28:28,881::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:28,881::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ddb22de-b2e5-4642-b700-d1ca500bf43b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:28,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ddb22de-b2e5-4642-b700-d1ca500bf43b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:28:28,882::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:28:28,882::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:28:28,882::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:28:28,882::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:28:28,882::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:28:28,882::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:28:28,882::task::993::Storage.TaskManager.Task::(_decref) Task=`6ddb22de-b2e5-4642-b700-d1ca500bf43b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:29,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:29,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117234 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:31,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:31,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114309 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:33,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:33,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116479 s, 87.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:28:34,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cde873e-8a35-44f5-92ab-df9d1c425a7a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:34,414::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:28:34,414::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000215725', 'lastCheck': '7.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185931', 'lastCheck': '7.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:34,414::task::1191::Storage.TaskManager.Task::(prepare) Task=`2cde873e-8a35-44f5-92ab-df9d1c425a7a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000215725', 'lastCheck': '7.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185931', 'lastCheck': '7.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:34,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cde873e-8a35-44f5-92ab-df9d1c425a7a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:34,414::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:34,414::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:34,414::task::993::Storage.TaskManager.Task::(_decref) Task=`2cde873e-8a35-44f5-92ab-df9d1c425a7a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:35,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:35,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012195 s, 84.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) (556) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:36,923::__init__::225::IOProcess::(_processLogs) (556) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:36,924::__init__::225::IOProcess::(_processLogs) (556) Building response >Thread-17::DEBUG::2014-08-22 09:28:36,924::__init__::225::IOProcess::(_processLogs) (556) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:36,924::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:28:36,925::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:28:36,931::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000249635 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:36,932::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:36,932::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:36,932::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:36,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:36,933::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:36,933::__init__::225::IOProcess::(_processLogs) (557) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:36,934::__init__::225::IOProcess::(_processLogs) (557) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:36,934::__init__::225::IOProcess::(_processLogs) (557) Building response >Thread-17::DEBUG::2014-08-22 09:28:36,934::__init__::225::IOProcess::(_processLogs) (557) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:36,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:36,934::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) (558) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:37,247::__init__::225::IOProcess::(_processLogs) (558) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:37,248::__init__::225::IOProcess::(_processLogs) (558) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:37,248::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:28:37,248::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:37,249::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:28:37,255::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000369685 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:37,255::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:37,255::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:37,256::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:37,256::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:37,256::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:37,256::__init__::225::IOProcess::(_processLogs) (559) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:37,256::__init__::225::IOProcess::(_processLogs) (559) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:37,257::__init__::225::IOProcess::(_processLogs) (559) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:37,257::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:37,257::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:37,258::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:28:37,258::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:37,258::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:37,258::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) (560) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) (560) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) (560) Building response >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) (560) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:28:37,259::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) (561) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) (561) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) (561) Building response >Thread-17::DEBUG::2014-08-22 09:28:37,260::__init__::225::IOProcess::(_processLogs) (561) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:37,261::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:37,261::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:37,261::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:28:37,261::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:37,261::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:37,261::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:37,262::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:37,262::__init__::225::IOProcess::(_processLogs) (562) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:37,262::__init__::225::IOProcess::(_processLogs) (562) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:37,262::__init__::225::IOProcess::(_processLogs) (562) Building response >Thread-17::DEBUG::2014-08-22 09:28:37,262::__init__::225::IOProcess::(_processLogs) (562) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:37,262::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:37,263::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:28:37,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:37,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117956 s, 86.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:28:38,934::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:28:38,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`253c7808-2adf-4a68-86d3-325638768705`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:38,934::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:28:38,934::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:38,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`253c7808-2adf-4a68-86d3-325638768705`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:38,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`253c7808-2adf-4a68-86d3-325638768705`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:38,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:38,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:38,935::task::993::Storage.TaskManager.Task::(_decref) Task=`253c7808-2adf-4a68-86d3-325638768705`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:28:38,988::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:28:38,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`67dc7f2b-4153-4fdf-8705-3ba92536ac1b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:28:38,988::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:28:38,988::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7ef6366e-fc18-4eee-9832-98e6b61015f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:28:38,988::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:28:38,989::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:28:38,989::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7ef6366e-fc18-4eee-9832-98e6b61015f1`::Granted request >Thread-46::DEBUG::2014-08-22 09:28:38,989::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`67dc7f2b-4153-4fdf-8705-3ba92536ac1b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:28:38,989::task::993::Storage.TaskManager.Task::(_decref) Task=`67dc7f2b-4153-4fdf-8705-3ba92536ac1b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:28:38,991::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:38,991::task::1191::Storage.TaskManager.Task::(prepare) Task=`67dc7f2b-4153-4fdf-8705-3ba92536ac1b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:38,991::task::595::Storage.TaskManager.Task::(_updateState) Task=`67dc7f2b-4153-4fdf-8705-3ba92536ac1b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:28:38,991::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:28:38,991::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:28:38,991::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:28:38,992::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:28:38,992::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:28:38,992::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:28:38,992::task::993::Storage.TaskManager.Task::(_decref) Task=`67dc7f2b-4153-4fdf-8705-3ba92536ac1b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:39,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:39,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116586 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:41,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:41,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136514 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:43,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:43,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116551 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:45,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:45,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131421 s, 77.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:46,934::__init__::225::IOProcess::(_processLogs) (563) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:46,935::__init__::225::IOProcess::(_processLogs) (563) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:46,935::__init__::225::IOProcess::(_processLogs) (563) Queuing response >Thread-21::DEBUG::2014-08-22 09:28:46,936::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:46,936::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:46,941::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:28:46,942::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00020197 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:46,942::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:46,943::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:46,943::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:46,943::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:46,943::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:46,943::__init__::225::IOProcess::(_processLogs) (564) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:46,943::__init__::225::IOProcess::(_processLogs) (564) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:46,944::__init__::225::IOProcess::(_processLogs) (564) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:46,944::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:46,945::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:47,262::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:47,262::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:47,263::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:47,263::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:47,263::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:47,263::__init__::225::IOProcess::(_processLogs) (565) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:47,263::__init__::225::IOProcess::(_processLogs) (565) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:47,263::__init__::225::IOProcess::(_processLogs) (565) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:47,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:28:47,264::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:47,265::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:28:47,270::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000297596 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) (566) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:47,271::__init__::225::IOProcess::(_processLogs) (566) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:47,272::__init__::225::IOProcess::(_processLogs) (566) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:47,273::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:47,273::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:47,273::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:28:47,273::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) (567) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) (567) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) (567) Building response >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) (567) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:47,274::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:47,275::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:28:47,275::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:47,275::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:47,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:47,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:47,275::__init__::225::IOProcess::(_processLogs) (568) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) (568) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) (568) Building response >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) (568) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:28:47,276::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:47,277::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:47,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:47,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:47,277::__init__::225::IOProcess::(_processLogs) (569) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:47,277::__init__::225::IOProcess::(_processLogs) (569) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:47,277::__init__::225::IOProcess::(_processLogs) (569) Building response >Thread-17::DEBUG::2014-08-22 09:28:47,278::__init__::225::IOProcess::(_processLogs) (569) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:47,278::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:47,278::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:28:47,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:47,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119832 s, 85.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:28:49,049::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:28:49,049::task::595::Storage.TaskManager.Task::(_updateState) Task=`d02a0ac7-6441-4d56-b5df-b13808b382bd`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:49,049::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:28:49,049::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:49,049::task::1191::Storage.TaskManager.Task::(prepare) Task=`d02a0ac7-6441-4d56-b5df-b13808b382bd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:49,049::task::595::Storage.TaskManager.Task::(_updateState) Task=`d02a0ac7-6441-4d56-b5df-b13808b382bd`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:49,050::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:49,050::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:49,050::task::993::Storage.TaskManager.Task::(_decref) Task=`d02a0ac7-6441-4d56-b5df-b13808b382bd`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:28:49,083::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:28:49,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9511826-53cf-48b2-84dd-9f0c0d7cd3f1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:28:49,084::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:28:49,084::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`50e4f4db-c9ad-42df-9e81-e060f2a012df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:28:49,084::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:28:49,084::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:28:49,084::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`50e4f4db-c9ad-42df-9e81-e060f2a012df`::Granted request >Thread-46::DEBUG::2014-08-22 09:28:49,084::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e9511826-53cf-48b2-84dd-9f0c0d7cd3f1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:28:49,084::task::993::Storage.TaskManager.Task::(_decref) Task=`e9511826-53cf-48b2-84dd-9f0c0d7cd3f1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:28:49,086::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:49,086::task::1191::Storage.TaskManager.Task::(prepare) Task=`e9511826-53cf-48b2-84dd-9f0c0d7cd3f1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:49,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9511826-53cf-48b2-84dd-9f0c0d7cd3f1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:28:49,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:28:49,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:28:49,087::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:28:49,087::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:28:49,087::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:28:49,087::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:28:49,087::task::993::Storage.TaskManager.Task::(_decref) Task=`e9511826-53cf-48b2-84dd-9f0c0d7cd3f1`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:28:49,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed69cd3b-312d-4efd-bcdb-dcfc262a970e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:49,624::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:28:49,625::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00020197', 'lastCheck': '2.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000297596', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:49,625::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed69cd3b-312d-4efd-bcdb-dcfc262a970e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00020197', 'lastCheck': '2.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000297596', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:28:49,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed69cd3b-312d-4efd-bcdb-dcfc262a970e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:49,625::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:49,625::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:49,625::task::993::Storage.TaskManager.Task::(_decref) Task=`ed69cd3b-312d-4efd-bcdb-dcfc262a970e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:49,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:49,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116856 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:51,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:51,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121956 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:53,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:53,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011625 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:28:55,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:55,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115042 s, 89.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:56,945::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:56,945::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:56,945::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:56,945::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:56,946::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:56,946::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:56,946::__init__::225::IOProcess::(_processLogs) (570) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:56,946::__init__::225::IOProcess::(_processLogs) (570) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:56,946::__init__::225::IOProcess::(_processLogs) (570) Queuing response >Thread-21::DEBUG::2014-08-22 09:28:56,947::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:56,947::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:56,953::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:28:56,954::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000171384 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) (571) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:56,955::__init__::225::IOProcess::(_processLogs) (571) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:56,956::__init__::225::IOProcess::(_processLogs) (571) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:56,957::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:56,957::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) (572) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:57,278::__init__::225::IOProcess::(_processLogs) (572) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:57,279::__init__::225::IOProcess::(_processLogs) (572) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:57,279::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:28:57,279::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:28:57,280::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:28:57,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214837 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) (573) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:28:57,286::__init__::225::IOProcess::(_processLogs) (573) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:28:57,287::__init__::225::IOProcess::(_processLogs) (573) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:57,288::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:57,288::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:28:57,288::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:28:57,288::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:57,288::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) (574) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) (574) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) (574) Building response >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) (574) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:28:57,289::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:57,290::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:57,290::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:57,290::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:57,290::__init__::225::IOProcess::(_processLogs) (575) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:57,290::__init__::225::IOProcess::(_processLogs) (575) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:57,290::__init__::225::IOProcess::(_processLogs) (575) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) (576) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:28:57,291::__init__::225::IOProcess::(_processLogs) (576) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:28:57,292::__init__::225::IOProcess::(_processLogs) (576) Queuing response >Thread-17::DEBUG::2014-08-22 09:28:57,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:28:57,292::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:28:57,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:28:57,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012103 s, 84.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:28:59,144::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-47::DEBUG::2014-08-22 09:28:59,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`bde7c3a1-1b04-4e9b-9e68-1ac9adb1ff72`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:28:59,145::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:28:59,145::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:59,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`bde7c3a1-1b04-4e9b-9e68-1ac9adb1ff72`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:28:59,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`bde7c3a1-1b04-4e9b-9e68-1ac9adb1ff72`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:28:59,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:28:59,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:28:59,146::task::993::Storage.TaskManager.Task::(_decref) Task=`bde7c3a1-1b04-4e9b-9e68-1ac9adb1ff72`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:28:59,179::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-46::DEBUG::2014-08-22 09:28:59,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ecc0768-8069-4826-bef8-71b7dca8b63f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:28:59,179::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:28:59,180::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b1cee22e-2769-4a2b-9157-d2f52ef8eafe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:28:59,180::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:28:59,180::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:28:59,180::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b1cee22e-2769-4a2b-9157-d2f52ef8eafe`::Granted request >Thread-46::DEBUG::2014-08-22 09:28:59,180::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2ecc0768-8069-4826-bef8-71b7dca8b63f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:28:59,180::task::993::Storage.TaskManager.Task::(_decref) Task=`2ecc0768-8069-4826-bef8-71b7dca8b63f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:28:59,182::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:59,182::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ecc0768-8069-4826-bef8-71b7dca8b63f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:28:59,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ecc0768-8069-4826-bef8-71b7dca8b63f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:28:59,182::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:28:59,183::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:28:59,183::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:28:59,183::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:28:59,183::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:28:59,183::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:28:59,183::task::993::Storage.TaskManager.Task::(_decref) Task=`2ecc0768-8069-4826-bef8-71b7dca8b63f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:28:59,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:00,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115849 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:02,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:02,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230395 s, 44.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:04,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:04,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115168 s, 88.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:04,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`42744141-6c25-499b-831d-21128930cc3a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:04,841::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:29:04,841::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000171384', 'lastCheck': '7.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214837', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:04,841::task::1191::Storage.TaskManager.Task::(prepare) Task=`42744141-6c25-499b-831d-21128930cc3a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000171384', 'lastCheck': '7.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214837', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:04,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`42744141-6c25-499b-831d-21128930cc3a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:04,841::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:04,841::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:04,841::task::993::Storage.TaskManager.Task::(_decref) Task=`42744141-6c25-499b-831d-21128930cc3a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:29:06,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:06,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115117 s, 89.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:06,957::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:06,957::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:06,958::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:06,958::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:06,958::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:06,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:06,958::__init__::225::IOProcess::(_processLogs) (577) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:06,958::__init__::225::IOProcess::(_processLogs) (577) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:06,959::__init__::225::IOProcess::(_processLogs) DEBUG|(577) Building response >Thread-21::DEBUG::2014-08-22 09:29:06,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:06,960::__init__::225::IOProcess::(_processLogs) (577) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:06,965::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:06,966::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:29:06,966::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000319305 s, 1.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:06,967::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:06,967::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) (578) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) (578) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) (578) Building response >Thread-17::DEBUG::2014-08-22 09:29:06,968::__init__::225::IOProcess::(_processLogs) (578) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:06,969::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:29:06,970::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:07,293::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:07,293::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:07,293::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:07,294::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:29:07,294::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:07,294::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:07,300::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:29:07,301::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198251 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:07,301::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:07,302::__init__::225::IOProcess::(_processLogs) (579) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:07,302::__init__::225::IOProcess::(_processLogs) (579) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:07,302::__init__::225::IOProcess::(_processLogs) (579) Building response >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) (579) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:07,303::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:07,304::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:07,304::__init__::225::IOProcess::(_processLogs) (580) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:07,304::__init__::225::IOProcess::(_processLogs) (580) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:07,304::__init__::225::IOProcess::(_processLogs) DEBUG|(580) Building response >Thread-17::DEBUG::2014-08-22 09:29:07,305::__init__::225::IOProcess::(_processLogs) (580) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:07,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:07,305::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:07,305::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:07,305::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:29:07,305::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:07,306::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:07,306::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:07,306::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:07,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:07,306::__init__::225::IOProcess::(_processLogs) (581) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) (581) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) (581) Building response >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) (581) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:29:07,307::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) (582) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) (582) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) (582) Building response >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) (582) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:07,308::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:07,309::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:07,309::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:07,309::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:29:07,309::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:07,309::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:07,309::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) (583) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) (583) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) (583) Building response >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) (583) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:07,310::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:07,311::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:29:08,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:08,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120156 s, 85.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:09,266::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:29:09,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`83bf5dd1-6f90-4955-b67b-e5dbf234f223`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:09,267::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:29:09,267::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:09,267::task::1191::Storage.TaskManager.Task::(prepare) Task=`83bf5dd1-6f90-4955-b67b-e5dbf234f223`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:09,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`83bf5dd1-6f90-4955-b67b-e5dbf234f223`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:09,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:09,268::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:09,268::task::993::Storage.TaskManager.Task::(_decref) Task=`83bf5dd1-6f90-4955-b67b-e5dbf234f223`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:29:09,308::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:29:09,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`7710518b-37d7-4645-a3fe-0c3f01326284`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:29:09,308::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:29:09,308::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b318f612-7b1d-4e26-b6e2-016b1fba8137`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:29:09,309::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:29:09,309::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:29:09,309::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b318f612-7b1d-4e26-b6e2-016b1fba8137`::Granted request >Thread-46::DEBUG::2014-08-22 09:29:09,309::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7710518b-37d7-4645-a3fe-0c3f01326284`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:29:09,309::task::993::Storage.TaskManager.Task::(_decref) Task=`7710518b-37d7-4645-a3fe-0c3f01326284`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:29:09,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:09,311::task::1191::Storage.TaskManager.Task::(prepare) Task=`7710518b-37d7-4645-a3fe-0c3f01326284`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:09,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`7710518b-37d7-4645-a3fe-0c3f01326284`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:29:09,311::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:29:09,311::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:29:09,312::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:29:09,312::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:29:09,312::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:29:09,312::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:29:09,312::task::993::Storage.TaskManager.Task::(_decref) Task=`7710518b-37d7-4645-a3fe-0c3f01326284`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:29:10,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:10,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115439 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:12,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:12,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114893 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:14,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:14,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117633 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:16,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:16,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011705 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:16,970::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:16,970::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) (584) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:16,971::__init__::225::IOProcess::(_processLogs) (584) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:16,972::__init__::225::IOProcess::(_processLogs) DEBUG|(584) Building response >Thread-21::DEBUG::2014-08-22 09:29:16,973::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:16,973::__init__::225::IOProcess::(_processLogs) (584) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:16,978::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:29:16,979::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000185776 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:16,979::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) (585) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) (585) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:16,981::__init__::225::IOProcess::(_processLogs) (585) Building response >Thread-17::DEBUG::2014-08-22 09:29:16,982::__init__::225::IOProcess::(_processLogs) (585) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:16,982::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:29:16,983::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:17,310::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:17,310::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:17,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) (586) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) (586) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:17,311::__init__::225::IOProcess::(_processLogs) DEBUG|(586) Building response >Thread-17::DEBUG::2014-08-22 09:29:17,312::__init__::225::IOProcess::(_processLogs) (586) Queuing response >Thread-22::DEBUG::2014-08-22 09:29:17,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:17,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:17,318::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:29:17,319::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000186205 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:17,319::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:17,319::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:17,319::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:17,319::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:17,319::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:17,319::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:17,320::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:17,320::__init__::225::IOProcess::(_processLogs) (587) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:17,320::__init__::225::IOProcess::(_processLogs) (587) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:17,320::__init__::225::IOProcess::(_processLogs) DEBUG|(587) Building response >Thread-17::DEBUG::2014-08-22 09:29:17,321::__init__::225::IOProcess::(_processLogs) (587) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:17,321::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:17,321::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:17,321::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:17,321::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:29:17,322::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:17,322::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:17,322::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:17,322::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:17,322::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) (588) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) (588) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) (588) Building response >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) (588) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:29:17,323::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) (589) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) (589) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) (589) Building response >Thread-17::DEBUG::2014-08-22 09:29:17,324::__init__::225::IOProcess::(_processLogs) (589) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:17,325::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:17,325::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:17,325::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:17,325::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:29:17,325::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:17,325::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:17,326::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:17,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:17,326::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:17,326::__init__::225::IOProcess::(_processLogs) (590) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:17,326::__init__::225::IOProcess::(_processLogs) (590) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:17,326::__init__::225::IOProcess::(_processLogs) (590) Building response >Thread-17::DEBUG::2014-08-22 09:29:17,327::__init__::225::IOProcess::(_processLogs) (590) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:17,327::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:17,327::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:29:18,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:18,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119358 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:19,363::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-47::DEBUG::2014-08-22 09:29:19,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca66a615-2ad0-4b82-82f0-23ded7ddac07`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:19,364::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:29:19,364::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:19,364::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca66a615-2ad0-4b82-82f0-23ded7ddac07`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:19,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca66a615-2ad0-4b82-82f0-23ded7ddac07`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:19,364::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:19,364::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:19,364::task::993::Storage.TaskManager.Task::(_decref) Task=`ca66a615-2ad0-4b82-82f0-23ded7ddac07`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:29:19,402::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-46::DEBUG::2014-08-22 09:29:19,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`55d400d0-c080-4795-92c2-0ff177566e94`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:29:19,403::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:29:19,403::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e3df5ce0-5809-4d9e-ac62-336f9b43c92b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:29:19,403::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:29:19,403::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:29:19,403::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e3df5ce0-5809-4d9e-ac62-336f9b43c92b`::Granted request >Thread-46::DEBUG::2014-08-22 09:29:19,404::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`55d400d0-c080-4795-92c2-0ff177566e94`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:29:19,404::task::993::Storage.TaskManager.Task::(_decref) Task=`55d400d0-c080-4795-92c2-0ff177566e94`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:29:19,406::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:19,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`55d400d0-c080-4795-92c2-0ff177566e94`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:19,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`55d400d0-c080-4795-92c2-0ff177566e94`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:29:19,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:29:19,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:29:19,406::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:29:19,406::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:29:19,406::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:29:19,406::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:29:19,406::task::993::Storage.TaskManager.Task::(_decref) Task=`55d400d0-c080-4795-92c2-0ff177566e94`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:29:20,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f27464a-354b-4a82-b942-0446bd379b72`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:20,028::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:29:20,028::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000185776', 'lastCheck': '3.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000186205', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:20,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f27464a-354b-4a82-b942-0446bd379b72`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000185776', 'lastCheck': '3.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000186205', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:20,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f27464a-354b-4a82-b942-0446bd379b72`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:20,029::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:20,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:20,029::task::993::Storage.TaskManager.Task::(_decref) Task=`2f27464a-354b-4a82-b942-0446bd379b72`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:29:20,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:20,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117525 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:22,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:22,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116354 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:24,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:24,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117989 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:26,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:26,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139237 s, 73.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:26,984::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:26,985::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:26,985::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:26,985::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:29:26,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:26,987::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:26,995::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:26,996::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:29:26,996::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000156781 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:26,997::__init__::225::IOProcess::(_processLogs) (591) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:26,998::__init__::225::IOProcess::(_processLogs) (591) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:26,998::__init__::225::IOProcess::(_processLogs) (591) Building response >Thread-17::DEBUG::2014-08-22 09:29:26,998::__init__::225::IOProcess::(_processLogs) (591) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:26,998::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:26,998::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:26,998::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) (592) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) (592) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) (592) Building response >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) (592) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:26,999::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:27,000::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:27,326::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) (593) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:27,327::__init__::225::IOProcess::(_processLogs) (593) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:27,328::__init__::225::IOProcess::(_processLogs) DEBUG|(593) Building response >Thread-17::DEBUG::2014-08-22 09:29:27,329::__init__::225::IOProcess::(_processLogs) (593) Queuing response >Thread-22::DEBUG::2014-08-22 09:29:27,329::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:27,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:27,335::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:29:27,336::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199775 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:27,337::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:27,337::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:27,337::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:27,338::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:27,338::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:27,338::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:27,338::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:27,338::__init__::225::IOProcess::(_processLogs) (594) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:27,338::__init__::225::IOProcess::(_processLogs) (594) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) (594) Building response >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) (594) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:29:27,339::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) (595) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) (595) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) (595) Building response >Thread-17::DEBUG::2014-08-22 09:29:27,340::__init__::225::IOProcess::(_processLogs) (595) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:27,341::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:27,341::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:27,341::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:27,341::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:29:27,341::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:27,341::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) (596) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) (596) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) (596) Building response >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) (596) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:27,342::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:27,343::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:27,343::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:29:27,343::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:27,343::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:27,343::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) (597) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) (597) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) (597) Building response >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) (597) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:27,344::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:29:28,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:28,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120701 s, 84.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:29,448::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:29:29,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d9f2ad5-73fc-438f-85ad-03752b4f190f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:29,448::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:29:29,449::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:29,449::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d9f2ad5-73fc-438f-85ad-03752b4f190f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:29,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d9f2ad5-73fc-438f-85ad-03752b4f190f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:29,449::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:29,449::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:29,449::task::993::Storage.TaskManager.Task::(_decref) Task=`6d9f2ad5-73fc-438f-85ad-03752b4f190f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:29:29,488::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:29:29,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cc5f5f7-24ce-475c-abc8-e72f0acb0886`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:29:29,488::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:29:29,489::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`46f52216-c5a9-4b52-995f-975cac4e56d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:29:29,489::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:29:29,489::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:29:29,489::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`46f52216-c5a9-4b52-995f-975cac4e56d8`::Granted request >Thread-46::DEBUG::2014-08-22 09:29:29,489::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2cc5f5f7-24ce-475c-abc8-e72f0acb0886`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:29:29,489::task::993::Storage.TaskManager.Task::(_decref) Task=`2cc5f5f7-24ce-475c-abc8-e72f0acb0886`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:29:29,491::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:29,491::task::1191::Storage.TaskManager.Task::(prepare) Task=`2cc5f5f7-24ce-475c-abc8-e72f0acb0886`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:29,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cc5f5f7-24ce-475c-abc8-e72f0acb0886`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:29:29,492::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:29:29,492::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:29:29,492::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:29:29,492::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:29:29,492::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:29:29,492::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:29:29,492::task::993::Storage.TaskManager.Task::(_decref) Task=`2cc5f5f7-24ce-475c-abc8-e72f0acb0886`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:29:30,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:30,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118776 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:32,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:32,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185487 s, 55.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:34,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:35,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133211 s, 76.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:35,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c6d94d3-8857-4e46-8de2-a2e553816a08`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:35,244::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:29:35,244::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000156781', 'lastCheck': '8.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199775', 'lastCheck': '7.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:35,244::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c6d94d3-8857-4e46-8de2-a2e553816a08`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000156781', 'lastCheck': '8.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199775', 'lastCheck': '7.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:35,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c6d94d3-8857-4e46-8de2-a2e553816a08`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:35,244::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:35,244::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:35,244::task::993::Storage.TaskManager.Task::(_decref) Task=`0c6d94d3-8857-4e46-8de2-a2e553816a08`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:29:37,001::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:37,001::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:37,001::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) (598) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) (598) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:37,002::__init__::225::IOProcess::(_processLogs) DEBUG|(598) Building response >Thread-17::DEBUG::2014-08-22 09:29:37,003::__init__::225::IOProcess::(_processLogs) (598) Queuing response >Thread-21::DEBUG::2014-08-22 09:29:37,003::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:37,003::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,009::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:29:37,009::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000226965 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) (599) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) (599) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:37,010::__init__::225::IOProcess::(_processLogs) (599) Building response >Thread-17::DEBUG::2014-08-22 09:29:37,011::__init__::225::IOProcess::(_processLogs) (599) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:37,011::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,012::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:29:37,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:37,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117031 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:37,344::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:37,344::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:37,345::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-22::DEBUG::2014-08-22 09:29:37,346::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:37,346::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:37,351::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,352::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:29:37,352::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000150737 s, 3.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:37,353::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,353::__init__::225::IOProcess::(_processLogs) (600) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:37,353::__init__::225::IOProcess::(_processLogs) (600) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:37,353::__init__::225::IOProcess::(_processLogs) (600) Building response >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) (600) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,354::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:37,355::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,355::__init__::225::IOProcess::(_processLogs) (601) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:37,355::__init__::225::IOProcess::(_processLogs) (601) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:37,355::__init__::225::IOProcess::(_processLogs) (601) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:37,356::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,356::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:37,356::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:29:37,356::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) (602) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) (602) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) (602) Building response >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) (602) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,357::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:37,358::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:29:37,358::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:37,358::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,358::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:37,358::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,358::__init__::225::IOProcess::(_processLogs) (603) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) (603) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) (603) Building response >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) (603) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:29:37,359::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) (604) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) (604) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) (604) Building response >Thread-17::DEBUG::2014-08-22 09:29:37,360::__init__::225::IOProcess::(_processLogs) (604) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:37,361::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:37,361::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:29:39,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:39,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118929 s, 86.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:39,542::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:29:39,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ee733fa-7f31-406b-8201-9c23ff29e9eb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:39,542::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:29:39,543::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:39,543::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ee733fa-7f31-406b-8201-9c23ff29e9eb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:39,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ee733fa-7f31-406b-8201-9c23ff29e9eb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:39,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:39,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:39,543::task::993::Storage.TaskManager.Task::(_decref) Task=`1ee733fa-7f31-406b-8201-9c23ff29e9eb`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:29:39,567::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:29:39,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`f26ac706-2378-4bac-9a95-1b16f5a733ed`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:29:39,567::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:29:39,567::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`356d4e0e-dec5-46cb-b661-94e8efef0a50`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:29:39,567::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:29:39,567::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:29:39,567::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`356d4e0e-dec5-46cb-b661-94e8efef0a50`::Granted request >Thread-46::DEBUG::2014-08-22 09:29:39,568::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f26ac706-2378-4bac-9a95-1b16f5a733ed`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:29:39,568::task::993::Storage.TaskManager.Task::(_decref) Task=`f26ac706-2378-4bac-9a95-1b16f5a733ed`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:29:39,570::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:39,570::task::1191::Storage.TaskManager.Task::(prepare) Task=`f26ac706-2378-4bac-9a95-1b16f5a733ed`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:39,570::task::595::Storage.TaskManager.Task::(_updateState) Task=`f26ac706-2378-4bac-9a95-1b16f5a733ed`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:29:39,570::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:29:39,570::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:29:39,570::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:29:39,570::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:29:39,570::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:29:39,571::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:29:39,571::task::993::Storage.TaskManager.Task::(_decref) Task=`f26ac706-2378-4bac-9a95-1b16f5a733ed`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:29:41,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:41,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118454 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:43,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:43,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116355 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:45,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:45,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011716 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) (605) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:47,013::__init__::225::IOProcess::(_processLogs) (605) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:47,014::__init__::225::IOProcess::(_processLogs) (605) Queuing response >Thread-21::DEBUG::2014-08-22 09:29:47,015::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:47,015::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:47,021::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:29:47,022::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000188064 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:47,022::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:47,022::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:47,023::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:47,023::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:47,023::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,023::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,023::__init__::225::IOProcess::(_processLogs) (606) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:47,023::__init__::225::IOProcess::(_processLogs) (606) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:47,024::__init__::225::IOProcess::(_processLogs) (606) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:47,025::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:47,025::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:29:47,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:47,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119821 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) (607) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:47,361::__init__::225::IOProcess::(_processLogs) (607) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:47,362::__init__::225::IOProcess::(_processLogs) (607) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:47,362::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:29:47,362::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:47,363::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:29:47,369::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000219997 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:47,369::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) (608) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) (608) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) (608) Building response >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) (608) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:47,370::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:47,371::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:29:47,371::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) (609) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) (609) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) (609) Building response >Thread-17::DEBUG::2014-08-22 09:29:47,372::__init__::225::IOProcess::(_processLogs) (609) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:47,373::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:47,373::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:47,373::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:29:47,373::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:47,373::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) (610) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) (610) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) (610) Building response >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) (610) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:29:47,374::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:47,375::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:47,375::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:47,375::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:47,375::__init__::225::IOProcess::(_processLogs) (611) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:47,376::__init__::225::IOProcess::(_processLogs) (611) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:47,376::__init__::225::IOProcess::(_processLogs) (611) Building response >Thread-17::DEBUG::2014-08-22 09:29:47,376::__init__::225::IOProcess::(_processLogs) (611) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:47,376::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:47,376::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:29:49,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:49,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119407 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:49,620::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:29:49,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b2372f1-74a3-46b6-a387-f9c8a0943abc`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:49,620::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:29:49,621::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:49,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b2372f1-74a3-46b6-a387-f9c8a0943abc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:49,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b2372f1-74a3-46b6-a387-f9c8a0943abc`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:49,621::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:49,621::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:49,621::task::993::Storage.TaskManager.Task::(_decref) Task=`9b2372f1-74a3-46b6-a387-f9c8a0943abc`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:29:49,655::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:29:49,655::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcce7851-a722-4b16-a638-4bcb77933deb`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:29:49,655::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:29:49,655::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ccfcf28e-3c84-4219-9ae7-360956005cb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:29:49,655::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:29:49,655::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:29:49,656::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ccfcf28e-3c84-4219-9ae7-360956005cb8`::Granted request >Thread-46::DEBUG::2014-08-22 09:29:49,656::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dcce7851-a722-4b16-a638-4bcb77933deb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:29:49,656::task::993::Storage.TaskManager.Task::(_decref) Task=`dcce7851-a722-4b16-a638-4bcb77933deb`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:29:49,658::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:49,658::task::1191::Storage.TaskManager.Task::(prepare) Task=`dcce7851-a722-4b16-a638-4bcb77933deb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:49,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcce7851-a722-4b16-a638-4bcb77933deb`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:29:49,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:29:49,658::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:29:49,658::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:29:49,658::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:29:49,659::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:29:49,659::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:29:49,659::task::993::Storage.TaskManager.Task::(_decref) Task=`dcce7851-a722-4b16-a638-4bcb77933deb`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:29:50,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ae2937f-19ff-4cfb-8413-5f2403192591`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:50,425::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:29:50,426::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000188064', 'lastCheck': '3.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219997', 'lastCheck': '3.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:50,426::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ae2937f-19ff-4cfb-8413-5f2403192591`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000188064', 'lastCheck': '3.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219997', 'lastCheck': '3.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:29:50,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ae2937f-19ff-4cfb-8413-5f2403192591`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:50,426::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:50,426::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:50,426::task::993::Storage.TaskManager.Task::(_decref) Task=`2ae2937f-19ff-4cfb-8413-5f2403192591`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:29:51,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:51,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121094 s, 84.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:53,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:53,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011587 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:55,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:55,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116121 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:57,025::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:57,025::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,025::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,025::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,025::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,025::__init__::225::IOProcess::(_processLogs) (612) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:57,026::__init__::225::IOProcess::(_processLogs) (612) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:57,026::__init__::225::IOProcess::(_processLogs) (612) Queuing response >Thread-21::DEBUG::2014-08-22 09:29:57,027::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:57,027::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,033::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:29:57,033::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000204618 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) (613) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:57,034::__init__::225::IOProcess::(_processLogs) (613) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:57,035::__init__::225::IOProcess::(_processLogs) (613) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:57,036::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,036::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:57,376::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:57,377::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-22::DEBUG::2014-08-22 09:29:57,377::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:29:57,377::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,383::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,383::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,383::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,383::__init__::225::IOProcess::(_processLogs) (614) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:57,383::__init__::225::IOProcess::(_processLogs) (614) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:57,383::__init__::225::IOProcess::(_processLogs) (614) Building response >Thread-17::DEBUG::2014-08-22 09:29:57,384::__init__::225::IOProcess::(_processLogs) (614) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:57,384::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,384::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:29:57,384::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000241588 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,385::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,386::__init__::225::IOProcess::(_processLogs) (615) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:29:57,386::__init__::225::IOProcess::(_processLogs) (615) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:29:57,386::__init__::225::IOProcess::(_processLogs) (615) Building response >Thread-17::DEBUG::2014-08-22 09:29:57,386::__init__::225::IOProcess::(_processLogs) (615) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:57,387::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,387::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:29:57,387::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:57,387::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:29:57,387::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:57,388::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,388::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,388::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,388::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,388::__init__::225::IOProcess::(_processLogs) (616) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:57,388::__init__::225::IOProcess::(_processLogs) (616) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) (616) Building response >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) (616) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,389::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) (617) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) (617) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) (617) Building response >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) (617) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,390::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:29:57,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:29:57,391::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:29:57,391::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:29:57,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) (618) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) (618) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) (618) Building response >Thread-17::DEBUG::2014-08-22 09:29:57,392::__init__::225::IOProcess::(_processLogs) (618) Queuing response >Thread-17::DEBUG::2014-08-22 09:29:57,393::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:29:57,393::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:29:57,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:57,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121233 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:29:59,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:29:59,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011554 s, 88.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:29:59,736::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:29:59,736::task::595::Storage.TaskManager.Task::(_updateState) Task=`338968fb-6814-4626-9344-371df5dd74a0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:29:59,736::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:29:59,736::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:59,737::task::1191::Storage.TaskManager.Task::(prepare) Task=`338968fb-6814-4626-9344-371df5dd74a0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:29:59,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`338968fb-6814-4626-9344-371df5dd74a0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:29:59,737::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:29:59,737::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:29:59,737::task::993::Storage.TaskManager.Task::(_decref) Task=`338968fb-6814-4626-9344-371df5dd74a0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:29:59,768::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:29:59,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e7ec56b-77c2-47c5-9eb9-4158201d699f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:29:59,768::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:29:59,768::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`db4cffd6-f6b8-4472-9af9-20a2d8974d97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:29:59,768::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:29:59,769::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:29:59,769::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`db4cffd6-f6b8-4472-9af9-20a2d8974d97`::Granted request >Thread-46::DEBUG::2014-08-22 09:29:59,769::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1e7ec56b-77c2-47c5-9eb9-4158201d699f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:29:59,769::task::993::Storage.TaskManager.Task::(_decref) Task=`1e7ec56b-77c2-47c5-9eb9-4158201d699f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:29:59,771::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:59,771::task::1191::Storage.TaskManager.Task::(prepare) Task=`1e7ec56b-77c2-47c5-9eb9-4158201d699f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:29:59,771::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e7ec56b-77c2-47c5-9eb9-4158201d699f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:29:59,771::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:29:59,771::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:29:59,772::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:29:59,772::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:29:59,772::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:29:59,772::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:29:59,772::task::993::Storage.TaskManager.Task::(_decref) Task=`1e7ec56b-77c2-47c5-9eb9-4158201d699f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:01,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:01,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120415 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:03,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:03,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120028 s, 85.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:05,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8dd4c56-5d5d-4ca4-93a0-55d3fd2099bf`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:05,654::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:30:05,654::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000204618', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000241588', 'lastCheck': '8.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:05,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`d8dd4c56-5d5d-4ca4-93a0-55d3fd2099bf`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000204618', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000241588', 'lastCheck': '8.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:05,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8dd4c56-5d5d-4ca4-93a0-55d3fd2099bf`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:05,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:05,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:05,654::task::993::Storage.TaskManager.Task::(_decref) Task=`d8dd4c56-5d5d-4ca4-93a0-55d3fd2099bf`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:05,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:05,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116653 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:07,036::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:07,036::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) (619) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) (619) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) (619) Building response >Thread-17::DEBUG::2014-08-22 09:30:07,037::__init__::225::IOProcess::(_processLogs) (619) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,038::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:30:07,039::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:07,039::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:30:07,045::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000177557 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) (620) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:07,046::__init__::225::IOProcess::(_processLogs) (620) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:07,047::__init__::225::IOProcess::(_processLogs) (620) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,048::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:07,048::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:07,393::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:07,393::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,393::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,393::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:30:07,394::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:07,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,400::__init__::225::IOProcess::(_processLogs) (621) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:30:07,401::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000175369 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:07,401::__init__::225::IOProcess::(_processLogs) (621) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) (621) Building response >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) (621) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:07,403::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,404::__init__::225::IOProcess::(_processLogs) (622) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:07,404::__init__::225::IOProcess::(_processLogs) (622) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:07,404::__init__::225::IOProcess::(_processLogs) DEBUG|(622) Building response >Thread-17::DEBUG::2014-08-22 09:30:07,405::__init__::225::IOProcess::(_processLogs) (622) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,405::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:07,405::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:07,405::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:07,405::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:30:07,405::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,406::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,406::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:07,406::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:07,406::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,406::__init__::225::IOProcess::(_processLogs) (623) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:07,406::__init__::225::IOProcess::(_processLogs) (623) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) (623) Building response >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) (623) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:30:07,407::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) (624) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) (624) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:07,408::__init__::225::IOProcess::(_processLogs) (624) Building response >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) (624) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:07,409::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:07,410::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:07,410::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:07,410::__init__::225::IOProcess::(_processLogs) (625) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:07,410::__init__::225::IOProcess::(_processLogs) (625) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:07,410::__init__::225::IOProcess::(_processLogs) (625) Building response >Thread-17::DEBUG::2014-08-22 09:30:07,411::__init__::225::IOProcess::(_processLogs) (625) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:07,411::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:07,411::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:30:07,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:07,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118365 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:09,820::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:30:09,820::task::595::Storage.TaskManager.Task::(_updateState) Task=`4687af90-b701-420f-899e-a664c1b1b6e0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:09,820::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:30:09,820::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:09,820::task::1191::Storage.TaskManager.Task::(prepare) Task=`4687af90-b701-420f-899e-a664c1b1b6e0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:09,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`4687af90-b701-420f-899e-a664c1b1b6e0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:09,821::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:09,821::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:09,821::task::993::Storage.TaskManager.Task::(_decref) Task=`4687af90-b701-420f-899e-a664c1b1b6e0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:09,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-46::DEBUG::2014-08-22 09:30:09,854::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:30:09,854::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d8185e5-03bf-47e7-83ca-bc4a036fe1d1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:30:09,854::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:30:09,855::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3c604f96-f7a9-47b7-9d66-4d94fc35ee80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:30:09,855::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:30:09,855::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:30:09,855::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3c604f96-f7a9-47b7-9d66-4d94fc35ee80`::Granted request >Thread-46::DEBUG::2014-08-22 09:30:09,855::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4d8185e5-03bf-47e7-83ca-bc4a036fe1d1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:30:09,856::task::993::Storage.TaskManager.Task::(_decref) Task=`4d8185e5-03bf-47e7-83ca-bc4a036fe1d1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:30:09,858::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:09,858::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d8185e5-03bf-47e7-83ca-bc4a036fe1d1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:09,858::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d8185e5-03bf-47e7-83ca-bc4a036fe1d1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:30:09,858::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:30:09,858::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:30:09,858::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:30:09,859::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:30:09,860::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:30:09,860::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:30:09,861::task::993::Storage.TaskManager.Task::(_decref) Task=`4d8185e5-03bf-47e7-83ca-bc4a036fe1d1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:09,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167968 s, 61.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:11,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:11,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115836 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:13,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:13,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115708 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:16,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:16,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115288 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:17,048::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,048::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:17,048::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,048::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,049::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,049::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,049::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,049::__init__::225::IOProcess::(_processLogs) (626) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:17,049::__init__::225::IOProcess::(_processLogs) (626) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:17,049::__init__::225::IOProcess::(_processLogs) DEBUG|(626) Building response >Thread-21::DEBUG::2014-08-22 09:30:17,050::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:17,050::__init__::225::IOProcess::(_processLogs) (626) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:17,056::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:30:17,057::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000172291 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:17,057::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) (627) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:17,059::__init__::225::IOProcess::(_processLogs) (627) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:17,060::__init__::225::IOProcess::(_processLogs) DEBUG|(627) Building response >Thread-17::DEBUG::2014-08-22 09:30:17,061::__init__::225::IOProcess::(_processLogs) (627) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:17,061::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:17,061::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,411::__init__::225::IOProcess::(_processLogs) (628) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:17,412::__init__::225::IOProcess::(_processLogs) (628) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 09:30:17,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:17,417::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(628) Building response >Thread-17::DEBUG::2014-08-22 09:30:17,418::__init__::225::IOProcess::(_processLogs) (628) Queuing response >Thread-22::DEBUG::2014-08-22 09:30:17,419::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000169417 s, 3.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:17,419::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:17,420::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:17,420::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) (629) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:17,421::__init__::225::IOProcess::(_processLogs) (629) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:17,422::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(629) Building response >Thread-17::DEBUG::2014-08-22 09:30:17,422::__init__::225::IOProcess::(_processLogs) (629) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:17,422::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:17,423::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:17,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,423::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:30:17,423::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) (630) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) (630) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) (630) Building response >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) (630) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:17,424::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,425::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:30:17,425::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,425::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,425::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,425::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,425::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) (631) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) (631) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) (631) Building response >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) (631) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:30:17,426::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:17,427::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:17,427::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:17,427::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) (632) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) (632) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) (632) Building response >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) (632) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:17,428::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:30:18,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:18,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011794 s, 86.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:19,906::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:30:19,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a7d4356-bd64-4d2d-aab9-e9f40f987195`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:19,907::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:30:19,907::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:19,907::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a7d4356-bd64-4d2d-aab9-e9f40f987195`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:19,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a7d4356-bd64-4d2d-aab9-e9f40f987195`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:19,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:19,907::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:19,908::task::993::Storage.TaskManager.Task::(_decref) Task=`1a7d4356-bd64-4d2d-aab9-e9f40f987195`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:30:19,932::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:30:19,932::task::595::Storage.TaskManager.Task::(_updateState) Task=`13baa859-3a02-4f9a-9ad9-d6e42c1704db`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:30:19,932::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:30:19,932::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fff83fff-7a83-45a5-970e-36b81037a3a5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:30:19,932::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:30:19,933::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:30:19,933::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fff83fff-7a83-45a5-970e-36b81037a3a5`::Granted request >Thread-46::DEBUG::2014-08-22 09:30:19,933::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`13baa859-3a02-4f9a-9ad9-d6e42c1704db`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:30:19,933::task::993::Storage.TaskManager.Task::(_decref) Task=`13baa859-3a02-4f9a-9ad9-d6e42c1704db`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:30:19,935::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:19,935::task::1191::Storage.TaskManager.Task::(prepare) Task=`13baa859-3a02-4f9a-9ad9-d6e42c1704db`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:19,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`13baa859-3a02-4f9a-9ad9-d6e42c1704db`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:30:19,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:30:19,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:30:19,935::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:30:19,935::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:30:19,936::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:30:19,936::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:30:19,936::task::993::Storage.TaskManager.Task::(_decref) Task=`13baa859-3a02-4f9a-9ad9-d6e42c1704db`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:20,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:20,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116235 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:20,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`b037bd98-db72-4649-b595-093d4376d303`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:20,846::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:30:20,846::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000172291', 'lastCheck': '3.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000169417', 'lastCheck': '3.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:20,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`b037bd98-db72-4649-b595-093d4376d303`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000172291', 'lastCheck': '3.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000169417', 'lastCheck': '3.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:20,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`b037bd98-db72-4649-b595-093d4376d303`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:20,846::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:20,846::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:20,846::task::993::Storage.TaskManager.Task::(_decref) Task=`b037bd98-db72-4649-b595-093d4376d303`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:22,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:22,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116657 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:24,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:24,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116097 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:26,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:26,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117091 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:27,061::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,061::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:27,061::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) (633) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) (633) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:27,062::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(633) Building response >Thread-21::DEBUG::2014-08-22 09:30:27,063::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:27,063::__init__::225::IOProcess::(_processLogs) (633) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,069::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,069::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:30:27,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182696 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:27,070::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,070::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:27,070::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,070::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,070::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,070::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,071::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,071::__init__::225::IOProcess::(_processLogs) (634) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:27,071::__init__::225::IOProcess::(_processLogs) (634) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:27,071::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(634) Building response >Thread-17::DEBUG::2014-08-22 09:30:27,072::__init__::225::IOProcess::(_processLogs) (634) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,072::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,072::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:27,428::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,428::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:27,428::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,428::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) (635) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) (635) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) (635) Building response >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) (635) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,429::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,430::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:30:27,430::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:30:27,436::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000235505 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:27,437::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,437::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:27,437::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,437::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,437::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,438::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,438::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,438::__init__::225::IOProcess::(_processLogs) (636) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:27,438::__init__::225::IOProcess::(_processLogs) (636) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:27,439::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(636) Building response >Thread-17::DEBUG::2014-08-22 09:30:27,439::__init__::225::IOProcess::(_processLogs) (636) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,439::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,439::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:27,439::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,440::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:30:27,440::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,440::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,440::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,440::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,440::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) (637) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) (637) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) (637) Building response >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) (637) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:30:27,441::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) (638) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) (638) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) (638) Building response >Thread-17::DEBUG::2014-08-22 09:30:27,442::__init__::225::IOProcess::(_processLogs) (638) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,443::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,443::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:27,443::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:27,443::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:30:27,443::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:27,443::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) (639) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) (639) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) (639) Building response >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) (639) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:27,444::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:27,445::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:30:28,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:28,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119344 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:30,022::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:30:30,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6234e6d-bb43-467d-bda7-21fd12acac62`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:30,023::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:30:30,023::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:30,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6234e6d-bb43-467d-bda7-21fd12acac62`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:30,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6234e6d-bb43-467d-bda7-21fd12acac62`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:30,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:30,024::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:30,024::task::993::Storage.TaskManager.Task::(_decref) Task=`e6234e6d-bb43-467d-bda7-21fd12acac62`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:30:30,064::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:30:30,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b747288-7e1f-47b1-9031-fb4a487177bf`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:30:30,064::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:30:30,064::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ecb73f4b-cc48-40f9-a4eb-8eb55ebbda89`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:30:30,064::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:30:30,064::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:30:30,065::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ecb73f4b-cc48-40f9-a4eb-8eb55ebbda89`::Granted request >Thread-46::DEBUG::2014-08-22 09:30:30,065::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3b747288-7e1f-47b1-9031-fb4a487177bf`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:30:30,065::task::993::Storage.TaskManager.Task::(_decref) Task=`3b747288-7e1f-47b1-9031-fb4a487177bf`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:30:30,067::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:30,067::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b747288-7e1f-47b1-9031-fb4a487177bf`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:30,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b747288-7e1f-47b1-9031-fb4a487177bf`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:30:30,067::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:30:30,067::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:30:30,067::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:30:30,068::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:30:30,068::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:30:30,068::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:30:30,068::task::993::Storage.TaskManager.Task::(_decref) Task=`3b747288-7e1f-47b1-9031-fb4a487177bf`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:30,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:30,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116288 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:32,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:32,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0292208 s, 35.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:34,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:34,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116264 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:36,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`89963ce9-78e6-4fca-93af-984bccf7d0b7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:36,112::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:30:36,113::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182696', 'lastCheck': '9.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000235505', 'lastCheck': '8.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:36,113::task::1191::Storage.TaskManager.Task::(prepare) Task=`89963ce9-78e6-4fca-93af-984bccf7d0b7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182696', 'lastCheck': '9.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000235505', 'lastCheck': '8.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:36,113::task::595::Storage.TaskManager.Task::(_updateState) Task=`89963ce9-78e6-4fca-93af-984bccf7d0b7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:36,113::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:36,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:36,113::task::993::Storage.TaskManager.Task::(_decref) Task=`89963ce9-78e6-4fca-93af-984bccf7d0b7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:36,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:36,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013631 s, 75.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:37,073::__init__::225::IOProcess::(_processLogs) (640) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:37,074::__init__::225::IOProcess::(_processLogs) (640) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:37,074::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(640) Building response >Thread-21::DEBUG::2014-08-22 09:30:37,075::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:37,075::__init__::225::IOProcess::(_processLogs) (640) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,082::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:30:37,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000201206 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:37,082::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:37,083::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,083::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) (641) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:37,084::__init__::225::IOProcess::(_processLogs) (641) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:37,085::__init__::225::IOProcess::(_processLogs) DEBUG|(641) Building response >Thread-17::DEBUG::2014-08-22 09:30:37,085::__init__::225::IOProcess::(_processLogs) (641) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,085::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:37,085::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:37,445::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,445::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:37,446::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,446::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,446::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:30:37,447::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:37,447::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,453::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:30:37,454::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000174479 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:37,454::__init__::225::IOProcess::(_processLogs) (642) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:37,455::__init__::225::IOProcess::(_processLogs) (642) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:37,455::__init__::225::IOProcess::(_processLogs) (642) Building response >Thread-17::DEBUG::2014-08-22 09:30:37,455::__init__::225::IOProcess::(_processLogs) (642) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,455::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:37,455::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:37,455::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) (643) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:37,456::__init__::225::IOProcess::(_processLogs) (643) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:37,457::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|(643) Building response >Thread-17::DEBUG::2014-08-22 09:30:37,457::__init__::225::IOProcess::(_processLogs) (643) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,458::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:37,458::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:37,458::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,458::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:30:37,458::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,458::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,459::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:37,459::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) (644) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) (644) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) (644) Building response >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) (644) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,460::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:30:37,461::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,461::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,461::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:37,461::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,461::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:37,461::__init__::225::IOProcess::(_processLogs) (645) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) (645) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) (645) Building response >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) (645) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:30:37,462::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:37,463::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:37,463::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:37,463::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:37,463::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:37,463::__init__::225::IOProcess::(_processLogs) (646) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:37,464::__init__::225::IOProcess::(_processLogs) (646) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:37,464::__init__::225::IOProcess::(_processLogs) (646) Building response >Thread-17::DEBUG::2014-08-22 09:30:37,464::__init__::225::IOProcess::(_processLogs) (646) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:37,464::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:37,464::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:30:38,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:38,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117389 s, 87.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:40,428::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:30:40,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`0694e609-1234-4965-bc6e-bc62adf1c049`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:40,428::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:30:40,428::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:40,429::task::1191::Storage.TaskManager.Task::(prepare) Task=`0694e609-1234-4965-bc6e-bc62adf1c049`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:40,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`0694e609-1234-4965-bc6e-bc62adf1c049`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:40,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:40,429::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:40,429::task::993::Storage.TaskManager.Task::(_decref) Task=`0694e609-1234-4965-bc6e-bc62adf1c049`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:30:40,465::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:30:40,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d405da4-d005-4fea-bbc8-a374fb25acdb`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:30:40,465::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:30:40,465::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5608b9ed-aa7c-42c6-b496-0f30fc4f5170`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:30:40,465::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:30:40,465::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:30:40,466::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5608b9ed-aa7c-42c6-b496-0f30fc4f5170`::Granted request >Thread-46::DEBUG::2014-08-22 09:30:40,466::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6d405da4-d005-4fea-bbc8-a374fb25acdb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:30:40,466::task::993::Storage.TaskManager.Task::(_decref) Task=`6d405da4-d005-4fea-bbc8-a374fb25acdb`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:30:40,468::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:40,468::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d405da4-d005-4fea-bbc8-a374fb25acdb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:40,468::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d405da4-d005-4fea-bbc8-a374fb25acdb`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:30:40,468::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:30:40,468::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:30:40,468::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:30:40,468::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:30:40,468::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:30:40,469::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:30:40,469::task::993::Storage.TaskManager.Task::(_decref) Task=`6d405da4-d005-4fea-bbc8-a374fb25acdb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:40,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:40,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116742 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:42,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:42,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0127823 s, 80.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:44,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:44,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117166 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:46,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:46,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011744 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:47,086::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,086::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:47,086::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:47,086::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,086::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:47,086::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:47,087::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,087::__init__::225::IOProcess::(_processLogs) (647) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:47,087::__init__::225::IOProcess::(_processLogs) (647) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:47,087::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|(647) Building response >Thread-21::DEBUG::2014-08-22 09:30:47,088::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:47,088::__init__::225::IOProcess::(_processLogs) (647) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,094::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:30:47,095::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000186196 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:47,095::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:47,096::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,096::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) (648) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:47,097::__init__::225::IOProcess::(_processLogs) (648) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:47,098::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|(648) Building response >Thread-17::DEBUG::2014-08-22 09:30:47,099::__init__::225::IOProcess::(_processLogs) (648) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,099::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:47,099::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:47,464::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,464::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:47,465::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-22::DEBUG::2014-08-22 09:30:47,465::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:47,466::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,472::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:47,473::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:30:47,473::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000209671 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:47,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) (649) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) (649) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) (649) Building response >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) (649) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,475::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) (650) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:47,476::__init__::225::IOProcess::(_processLogs) (650) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:47,477::__init__::225::IOProcess::(_processLogs) DEBUG|(650) Building response >Thread-17::DEBUG::2014-08-22 09:30:47,477::__init__::225::IOProcess::(_processLogs) (650) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,478::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:47,478::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:47,478::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,478::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:30:47,478::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:47,478::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) (651) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) (651) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) (651) Building response >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) (651) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:47,479::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:47,480::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,480::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:30:47,480::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:47,480::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) (652) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) (652) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) (652) Building response >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) (652) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:47,481::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:47,482::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:30:47,482::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:47,482::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:47,482::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) (653) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) (653) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) (653) Building response >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) (653) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:47,483::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:30:48,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:48,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117525 s, 87.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:50,713::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:30:50,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`1153c59f-f073-4912-845c-17eacac5ddec`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:50,713::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:30:50,714::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:50,714::task::1191::Storage.TaskManager.Task::(prepare) Task=`1153c59f-f073-4912-845c-17eacac5ddec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:30:50,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`1153c59f-f073-4912-845c-17eacac5ddec`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:50,714::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:50,714::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:50,714::task::993::Storage.TaskManager.Task::(_decref) Task=`1153c59f-f073-4912-845c-17eacac5ddec`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:30:50,762::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:30:50,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ebeb073-9691-4c96-951b-a275c5a1bca9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:30:50,762::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:30:50,762::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a0b01d73-6839-4aaf-babd-4097b30645e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:30:50,762::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:30:50,763::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:30:50,763::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a0b01d73-6839-4aaf-babd-4097b30645e6`::Granted request >Thread-46::DEBUG::2014-08-22 09:30:50,763::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4ebeb073-9691-4c96-951b-a275c5a1bca9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:30:50,763::task::993::Storage.TaskManager.Task::(_decref) Task=`4ebeb073-9691-4c96-951b-a275c5a1bca9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:30:50,765::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:50,765::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ebeb073-9691-4c96-951b-a275c5a1bca9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:30:50,765::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ebeb073-9691-4c96-951b-a275c5a1bca9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:30:50,765::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:30:50,765::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:30:50,765::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:30:50,766::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:30:50,766::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:30:50,766::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:30:50,766::task::993::Storage.TaskManager.Task::(_decref) Task=`4ebeb073-9691-4c96-951b-a275c5a1bca9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:50,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:51,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119543 s, 85.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:30:51,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6c53479-cefb-4a68-8a2f-ad484779ba89`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:30:51,362::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:30:51,362::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000186196', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209671', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:51,362::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6c53479-cefb-4a68-8a2f-ad484779ba89`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000186196', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209671', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:30:51,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6c53479-cefb-4a68-8a2f-ad484779ba89`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:30:51,363::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:30:51,363::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:30:51,363::task::993::Storage.TaskManager.Task::(_decref) Task=`b6c53479-cefb-4a68-8a2f-ad484779ba89`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:30:53,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:53,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011551 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:30:55,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:55,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118749 s, 86.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:57,099::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:57,099::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:57,099::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) (654) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) (654) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:57,100::__init__::225::IOProcess::(_processLogs) DEBUG|(654) Building response >Thread-21::DEBUG::2014-08-22 09:30:57,102::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:57,102::__init__::225::IOProcess::(_processLogs) (654) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,107::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:30:57,108::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000185808 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:57,109::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,110::__init__::225::IOProcess::(_processLogs) (655) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:57,111::__init__::225::IOProcess::(_processLogs) (655) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:57,111::__init__::225::IOProcess::(_processLogs) (655) Building response >Thread-17::DEBUG::2014-08-22 09:30:57,112::__init__::225::IOProcess::(_processLogs) (655) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,112::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:57,112::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:30:57,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:57,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116066 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:57,483::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) (656) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) (656) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:57,484::__init__::225::IOProcess::(_processLogs) (656) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,485::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:30:57,485::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:30:57,486::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:30:57,493::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021073 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:30:57,495::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:30:57,495::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:57,495::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:30:57,495::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,496::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) (657) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) (657) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) (657) Building response >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) (657) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:30:57,497::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:57,498::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,498::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:57,498::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,498::__init__::225::IOProcess::(_processLogs) (658) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:57,498::__init__::225::IOProcess::(_processLogs) (658) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:57,498::__init__::225::IOProcess::(_processLogs) (658) Building response >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) (658) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) (659) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:57,499::__init__::225::IOProcess::(_processLogs) (659) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:57,500::__init__::225::IOProcess::(_processLogs) (659) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,501::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:30:57,501::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:30:57,501::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:30:57,502::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:30:57,502::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:30:57,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:30:57,502::__init__::225::IOProcess::(_processLogs) (660) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:30:57,502::__init__::225::IOProcess::(_processLogs) (660) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:30:57,502::__init__::225::IOProcess::(_processLogs) (660) Queuing response >Thread-17::DEBUG::2014-08-22 09:30:57,503::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:30:57,503::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:30:59,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:30:59,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118372 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:00,839::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:31:00,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a3c76a5-0792-4afb-ab6f-27003386fdc5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:00,839::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:31:00,839::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:00,840::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a3c76a5-0792-4afb-ab6f-27003386fdc5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:00,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a3c76a5-0792-4afb-ab6f-27003386fdc5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:00,840::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:00,840::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:00,840::task::993::Storage.TaskManager.Task::(_decref) Task=`1a3c76a5-0792-4afb-ab6f-27003386fdc5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:31:00,872::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:31:00,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`88872b14-17bf-4f9b-b95d-fe3b64c5c014`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:31:00,873::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:31:00,873::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e15f9532-0223-4898-a5d2-2a6e5903d7ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:31:00,873::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:31:00,873::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:31:00,873::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e15f9532-0223-4898-a5d2-2a6e5903d7ef`::Granted request >Thread-46::DEBUG::2014-08-22 09:31:00,873::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`88872b14-17bf-4f9b-b95d-fe3b64c5c014`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:31:00,874::task::993::Storage.TaskManager.Task::(_decref) Task=`88872b14-17bf-4f9b-b95d-fe3b64c5c014`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:31:00,875::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:00,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`88872b14-17bf-4f9b-b95d-fe3b64c5c014`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:00,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`88872b14-17bf-4f9b-b95d-fe3b64c5c014`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:31:00,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:31:00,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:31:00,876::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:31:00,876::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:31:00,876::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:31:00,876::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:31:00,876::task::993::Storage.TaskManager.Task::(_decref) Task=`88872b14-17bf-4f9b-b95d-fe3b64c5c014`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:01,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:01,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117069 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:03,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:03,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120435 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:05,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:05,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011699 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:06,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`309f8c41-bcdd-4907-9b5a-7de8b7e1212a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:06,532::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:31:06,532::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000185808', 'lastCheck': '9.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021073', 'lastCheck': '9.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:06,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`309f8c41-bcdd-4907-9b5a-7de8b7e1212a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000185808', 'lastCheck': '9.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021073', 'lastCheck': '9.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:06,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`309f8c41-bcdd-4907-9b5a-7de8b7e1212a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:06,532::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:06,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:06,533::task::993::Storage.TaskManager.Task::(_decref) Task=`309f8c41-bcdd-4907-9b5a-7de8b7e1212a`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:31:07,112::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:07,113::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:07,113::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,113::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:07,113::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,113::__init__::225::IOProcess::(_processLogs) (661) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:07,113::__init__::225::IOProcess::(_processLogs) (661) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:07,114::__init__::225::IOProcess::(_processLogs) (661) Queuing response >Thread-21::DEBUG::2014-08-22 09:31:07,114::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:07,114::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,121::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:31:07,121::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000183796 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:07,122::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:07,122::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:07,122::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:07,122::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,122::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:07,122::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,123::__init__::225::IOProcess::(_processLogs) (662) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:07,123::__init__::225::IOProcess::(_processLogs) (662) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:07,123::__init__::225::IOProcess::(_processLogs) DEBUG|(662) Building response >Thread-17::DEBUG::2014-08-22 09:31:07,124::__init__::225::IOProcess::(_processLogs) (662) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:07,124::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,124::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:31:07,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:07,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119251 s, 85.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:07,503::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:07,504::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:07,504::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:07,505::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-22::DEBUG::2014-08-22 09:31:07,505::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:07,505::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,511::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:31:07,512::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000215866 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:07,512::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,513::__init__::225::IOProcess::(_processLogs) (663) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:07,513::__init__::225::IOProcess::(_processLogs) (663) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) (663) Building response >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) (663) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:07,514::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:07,515::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:07,515::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) (664) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) (664) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) (664) Building response >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) (664) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:07,516::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:31:07,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:07,517::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:07,517::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,517::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:07,517::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,517::__init__::225::IOProcess::(_processLogs) (665) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) (665) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) (665) Building response >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) (665) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:31:07,518::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) (666) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) (666) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) (666) Building response >Thread-17::DEBUG::2014-08-22 09:31:07,519::__init__::225::IOProcess::(_processLogs) (666) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:07,520::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,520::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:07,520::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:07,520::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:31:07,520::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:07,520::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) (667) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) (667) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) (667) Building response >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) (667) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:07,521::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:07,522::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:31:09,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:09,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124015 s, 82.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:10,939::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:31:10,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`45672fbb-050a-44f7-b93b-9c1533b7e39e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:10,940::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:31:10,940::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:10,940::task::1191::Storage.TaskManager.Task::(prepare) Task=`45672fbb-050a-44f7-b93b-9c1533b7e39e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:10,940::task::595::Storage.TaskManager.Task::(_updateState) Task=`45672fbb-050a-44f7-b93b-9c1533b7e39e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:10,940::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:10,940::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:10,940::task::993::Storage.TaskManager.Task::(_decref) Task=`45672fbb-050a-44f7-b93b-9c1533b7e39e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:31:10,991::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:31:10,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`80677ba0-0ea6-4bc9-8699-b6d3b3fc4db5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:31:10,992::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:31:10,992::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`319a1a1c-1d79-40d8-bd66-8465ff4df7ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:31:10,992::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:31:10,992::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:31:10,992::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`319a1a1c-1d79-40d8-bd66-8465ff4df7ac`::Granted request >Thread-46::DEBUG::2014-08-22 09:31:10,992::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`80677ba0-0ea6-4bc9-8699-b6d3b3fc4db5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:31:10,993::task::993::Storage.TaskManager.Task::(_decref) Task=`80677ba0-0ea6-4bc9-8699-b6d3b3fc4db5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:31:10,995::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:10,995::task::1191::Storage.TaskManager.Task::(prepare) Task=`80677ba0-0ea6-4bc9-8699-b6d3b3fc4db5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:10,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`80677ba0-0ea6-4bc9-8699-b6d3b3fc4db5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:31:10,995::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:31:10,995::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:31:10,995::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:31:10,995::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:31:10,995::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:31:10,995::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:31:10,995::task::993::Storage.TaskManager.Task::(_decref) Task=`80677ba0-0ea6-4bc9-8699-b6d3b3fc4db5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:11,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:11,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116273 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:13,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:13,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121699 s, 84.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:15,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:15,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011671 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,125::__init__::225::IOProcess::(_processLogs) (668) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:17,126::__init__::225::IOProcess::(_processLogs) (668) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:17,126::__init__::225::IOProcess::(_processLogs) DEBUG|(668) Building response >Thread-21::DEBUG::2014-08-22 09:31:17,127::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:17,127::__init__::225::IOProcess::(_processLogs) (668) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:17,133::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:31:17,134::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000213004 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:17,134::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:17,135::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) (669) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:17,136::__init__::225::IOProcess::(_processLogs) (669) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:17,137::__init__::225::IOProcess::(_processLogs) DEBUG|(669) Building response >Thread-17::DEBUG::2014-08-22 09:31:17,138::__init__::225::IOProcess::(_processLogs) (669) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:17,138::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:17,138::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:17,521::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,521::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) (670) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) (670) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:17,522::__init__::225::IOProcess::(_processLogs) DEBUG|(670) Building response >Thread-17::DEBUG::2014-08-22 09:31:17,523::__init__::225::IOProcess::(_processLogs) (670) Queuing response >Thread-22::DEBUG::2014-08-22 09:31:17,523::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:17,524::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:17,530::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:31:17,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000208735 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,531::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,532::__init__::225::IOProcess::(_processLogs) (671) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:17,532::__init__::225::IOProcess::(_processLogs) (671) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:17,532::__init__::225::IOProcess::(_processLogs) DEBUG|(671) Building response >Thread-17::DEBUG::2014-08-22 09:31:17,533::__init__::225::IOProcess::(_processLogs) (671) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:17,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:17,533::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:17,533::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,533::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:31:17,533::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) (672) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) (672) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:17,534::__init__::225::IOProcess::(_processLogs) (672) Building response >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) (672) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,535::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) (673) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) (673) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) (673) Building response >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) (673) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:17,536::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:17,537::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:17,537::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:31:17,537::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:17,537::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:17,537::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) (674) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) (674) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) (674) Building response >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) (674) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:17,538::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:31:17,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:17,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117529 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:19,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:19,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011606 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:21,046::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:31:21,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`b729e0ed-4b10-4ec0-9585-c57c8bc455c8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:21,047::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:31:21,047::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:21,047::task::1191::Storage.TaskManager.Task::(prepare) Task=`b729e0ed-4b10-4ec0-9585-c57c8bc455c8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:21,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`b729e0ed-4b10-4ec0-9585-c57c8bc455c8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:21,047::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:21,047::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:21,047::task::993::Storage.TaskManager.Task::(_decref) Task=`b729e0ed-4b10-4ec0-9585-c57c8bc455c8`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:31:21,088::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:31:21,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`d138c787-f119-423b-ba9b-ee9c71ccb510`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:31:21,089::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:31:21,089::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`06686edc-aec7-4ac6-b938-069e65edbdd7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:31:21,089::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:31:21,089::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:31:21,089::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`06686edc-aec7-4ac6-b938-069e65edbdd7`::Granted request >Thread-46::DEBUG::2014-08-22 09:31:21,089::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d138c787-f119-423b-ba9b-ee9c71ccb510`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:31:21,090::task::993::Storage.TaskManager.Task::(_decref) Task=`d138c787-f119-423b-ba9b-ee9c71ccb510`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:31:21,091::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:21,092::task::1191::Storage.TaskManager.Task::(prepare) Task=`d138c787-f119-423b-ba9b-ee9c71ccb510`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:21,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`d138c787-f119-423b-ba9b-ee9c71ccb510`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:31:21,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:31:21,092::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:31:21,092::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:31:21,092::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:31:21,092::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:31:21,092::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:31:21,092::task::993::Storage.TaskManager.Task::(_decref) Task=`d138c787-f119-423b-ba9b-ee9c71ccb510`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:21,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:21,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115768 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:21,798::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4872a91-1412-48a1-b061-01faba8e3bfd`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:21,799::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:31:21,799::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000213004', 'lastCheck': '4.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000208735', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:21,799::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4872a91-1412-48a1-b061-01faba8e3bfd`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000213004', 'lastCheck': '4.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000208735', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:21,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4872a91-1412-48a1-b061-01faba8e3bfd`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:21,799::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:21,799::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:21,799::task::993::Storage.TaskManager.Task::(_decref) Task=`a4872a91-1412-48a1-b061-01faba8e3bfd`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:23,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:23,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117537 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:25,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:25,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116538 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:27,138::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,138::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:27,138::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,138::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:27,138::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,139::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:27,139::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,139::__init__::225::IOProcess::(_processLogs) (675) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:27,139::__init__::225::IOProcess::(_processLogs) (675) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:27,139::__init__::225::IOProcess::(_processLogs) DEBUG|(675) Building response >Thread-21::DEBUG::2014-08-22 09:31:27,140::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:27,140::__init__::225::IOProcess::(_processLogs) (675) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,147::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:31:27,148::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227468 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:27,148::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:27,149::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,150::__init__::225::IOProcess::(_processLogs) (676) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:27,150::__init__::225::IOProcess::(_processLogs) (676) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:27,150::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(676) Building response >Thread-17::DEBUG::2014-08-22 09:31:27,151::__init__::225::IOProcess::(_processLogs) (676) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,151::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:27,151::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:27,538::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,538::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:27,538::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,539::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:31:27,540::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:27,540::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,546::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:31:27,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000247957 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:27,547::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) (677) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) (677) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) (677) Building response >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) (677) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:27,549::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) (678) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:27,550::__init__::225::IOProcess::(_processLogs) (678) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:27,551::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(678) Building response >Thread-17::DEBUG::2014-08-22 09:31:27,551::__init__::225::IOProcess::(_processLogs) (678) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,551::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:27,552::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:27,552::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,552::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:31:27,552::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,552::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) (679) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) (679) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) (679) Building response >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) (679) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:27,553::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,554::__init__::225::IOProcess::(_processLogs) (680) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) (680) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) (680) Building response >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) (680) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:31:27,555::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:27,556::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:27,556::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:27,556::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:27,556::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:27,556::__init__::225::IOProcess::(_processLogs) (681) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:27,557::__init__::225::IOProcess::(_processLogs) (681) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:27,557::__init__::225::IOProcess::(_processLogs) (681) Building response >Thread-17::DEBUG::2014-08-22 09:31:27,557::__init__::225::IOProcess::(_processLogs) (681) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:27,557::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:27,557::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:31:27,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:27,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013961 s, 73.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:29,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:29,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118701 s, 86.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:31,141::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:31:31,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`7047808e-5598-44bb-a1d0-41ee90ce5053`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:31,142::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:31:31,142::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:31,142::task::1191::Storage.TaskManager.Task::(prepare) Task=`7047808e-5598-44bb-a1d0-41ee90ce5053`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:31,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`7047808e-5598-44bb-a1d0-41ee90ce5053`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:31,142::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:31,142::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:31,142::task::993::Storage.TaskManager.Task::(_decref) Task=`7047808e-5598-44bb-a1d0-41ee90ce5053`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:31:31,176::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:31:31,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`19c2cea0-7924-4fa0-9b89-a26b341c7d0c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:31:31,177::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:31:31,177::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4b90d753-489e-4ea7-9f34-737225fee108`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:31:31,177::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:31:31,177::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:31:31,177::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4b90d753-489e-4ea7-9f34-737225fee108`::Granted request >Thread-46::DEBUG::2014-08-22 09:31:31,178::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`19c2cea0-7924-4fa0-9b89-a26b341c7d0c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:31:31,178::task::993::Storage.TaskManager.Task::(_decref) Task=`19c2cea0-7924-4fa0-9b89-a26b341c7d0c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:31:31,180::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:31,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`19c2cea0-7924-4fa0-9b89-a26b341c7d0c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:31,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`19c2cea0-7924-4fa0-9b89-a26b341c7d0c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:31:31,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:31:31,180::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:31:31,180::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:31:31,180::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:31:31,180::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:31:31,180::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:31:31,180::task::993::Storage.TaskManager.Task::(_decref) Task=`19c2cea0-7924-4fa0-9b89-a26b341c7d0c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:31,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:32,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188717 s, 54.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:34,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:34,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117119 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:36,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:36,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116054 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:36,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc3078dd-b804-42a7-9cc8-9689ffb6b16d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:36,998::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:31:36,998::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227468', 'lastCheck': '9.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000247957', 'lastCheck': '9.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:36,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc3078dd-b804-42a7-9cc8-9689ffb6b16d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227468', 'lastCheck': '9.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000247957', 'lastCheck': '9.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:36,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc3078dd-b804-42a7-9cc8-9689ffb6b16d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:36,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:36,998::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:36,998::task::993::Storage.TaskManager.Task::(_decref) Task=`cc3078dd-b804-42a7-9cc8-9689ffb6b16d`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:31:37,151::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) (682) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:37,152::__init__::225::IOProcess::(_processLogs) (682) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:37,153::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(682) Building response >Thread-21::DEBUG::2014-08-22 09:31:37,153::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:37,154::__init__::225::IOProcess::(_processLogs) (682) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:31:37,160::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00021983 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:37,161::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:37,162::__init__::225::IOProcess::(_processLogs) (683) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:37,163::__init__::225::IOProcess::(_processLogs) (683) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:37,163::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(683) Building response >Thread-17::DEBUG::2014-08-22 09:31:37,164::__init__::225::IOProcess::(_processLogs) (683) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,164::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:37,164::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:37,557::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,557::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:37,557::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,557::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,558::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:31:37,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:37,558::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,564::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:31:37,565::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000147568 s, 3.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:37,565::__init__::225::IOProcess::(_processLogs) (684) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:37,566::__init__::225::IOProcess::(_processLogs) (684) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:37,566::__init__::225::IOProcess::(_processLogs) (684) Building response >Thread-17::DEBUG::2014-08-22 09:31:37,566::__init__::225::IOProcess::(_processLogs) (684) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:37,567::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,568::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:37,568::__init__::225::IOProcess::(_processLogs) (685) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:37,568::__init__::225::IOProcess::(_processLogs) (685) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:37,568::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(685) Building response >Thread-17::DEBUG::2014-08-22 09:31:37,569::__init__::225::IOProcess::(_processLogs) (685) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,569::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:37,569::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:37,569::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,569::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:31:37,569::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,570::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:37,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:37,570::__init__::225::IOProcess::(_processLogs) (686) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:37,570::__init__::225::IOProcess::(_processLogs) (686) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) (686) Building response >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) (686) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:31:37,571::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) (687) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) (687) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) (687) Building response >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) (687) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,572::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:37,573::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:37,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:37,573::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:31:37,573::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:37,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:37,574::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:37,574::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:37,574::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:37,574::__init__::225::IOProcess::(_processLogs) (688) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:37,574::__init__::225::IOProcess::(_processLogs) (688) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:37,574::__init__::225::IOProcess::(_processLogs) (688) Building response >Thread-17::DEBUG::2014-08-22 09:31:37,575::__init__::225::IOProcess::(_processLogs) (688) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:37,575::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:37,575::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:31:38,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:38,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119847 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:40,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:40,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118519 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:41,230::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-47::DEBUG::2014-08-22 09:31:41,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`18b20805-8e71-465f-bf3e-7547de276926`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:41,231::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:31:41,231::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:41,232::task::1191::Storage.TaskManager.Task::(prepare) Task=`18b20805-8e71-465f-bf3e-7547de276926`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:41,232::task::595::Storage.TaskManager.Task::(_updateState) Task=`18b20805-8e71-465f-bf3e-7547de276926`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:41,232::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:41,232::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:41,232::task::993::Storage.TaskManager.Task::(_decref) Task=`18b20805-8e71-465f-bf3e-7547de276926`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:31:41,262::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-46::DEBUG::2014-08-22 09:31:41,262::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c8930b0-ac7b-48ce-94b5-3da9e6c3549d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:31:41,262::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:31:41,262::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0170f7f2-b94d-4004-8ed3-a91ccd31a083`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:31:41,262::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:31:41,263::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:31:41,263::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0170f7f2-b94d-4004-8ed3-a91ccd31a083`::Granted request >Thread-46::DEBUG::2014-08-22 09:31:41,264::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0c8930b0-ac7b-48ce-94b5-3da9e6c3549d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:31:41,264::task::993::Storage.TaskManager.Task::(_decref) Task=`0c8930b0-ac7b-48ce-94b5-3da9e6c3549d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:31:41,266::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:41,266::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c8930b0-ac7b-48ce-94b5-3da9e6c3549d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:41,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c8930b0-ac7b-48ce-94b5-3da9e6c3549d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:31:41,266::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:31:41,266::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:31:41,266::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:31:41,266::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:31:41,266::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:31:41,266::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:31:41,267::task::993::Storage.TaskManager.Task::(_decref) Task=`0c8930b0-ac7b-48ce-94b5-3da9e6c3549d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:42,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:42,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115377 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:44,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:44,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011725 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:46,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:46,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117456 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:47,164::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) (689) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:47,165::__init__::225::IOProcess::(_processLogs) (689) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:47,166::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(689) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,166::__init__::225::IOProcess::(_processLogs) (689) Queuing response >Thread-21::DEBUG::2014-08-22 09:31:47,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:47,167::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,173::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:31:47,174::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000166207 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) (690) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:47,175::__init__::225::IOProcess::(_processLogs) (690) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:47,176::__init__::225::IOProcess::(_processLogs) (690) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,177::__init__::225::IOProcess::(_processLogs) (690) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:47,177::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,177::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:47,574::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,574::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:47,575::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,575::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:31:47,576::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:47,576::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,582::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:31:47,583::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000230101 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:47,583::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,584::__init__::225::IOProcess::(_processLogs) (691) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:47,584::__init__::225::IOProcess::(_processLogs) (691) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:47,584::__init__::225::IOProcess::(_processLogs) (691) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,584::__init__::225::IOProcess::(_processLogs) (691) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:47,584::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,584::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) (692) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:47,585::__init__::225::IOProcess::(_processLogs) (692) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:47,586::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|(692) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,587::__init__::225::IOProcess::(_processLogs) (692) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:47,587::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,587::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:47,587::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,587::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:31:47,587::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) (693) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) (693) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) (693) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) (693) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:47,588::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,589::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:47,589::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,589::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:31:47,589::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,589::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) (694) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) (694) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) (694) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) (694) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:47,590::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:31:47,591::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:47,591::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:47,591::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) (695) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) (695) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) (695) Building response >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) (695) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:47,592::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:31:48,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:48,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01205 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:50,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:50,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121829 s, 84.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:31:51,317::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:31:51,317::task::595::Storage.TaskManager.Task::(_updateState) Task=`629c18b9-6a34-422f-bcf6-f3f5aa6f3145`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:51,317::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:31:51,318::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:51,318::task::1191::Storage.TaskManager.Task::(prepare) Task=`629c18b9-6a34-422f-bcf6-f3f5aa6f3145`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:31:51,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`629c18b9-6a34-422f-bcf6-f3f5aa6f3145`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:51,318::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:51,318::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:51,318::task::993::Storage.TaskManager.Task::(_decref) Task=`629c18b9-6a34-422f-bcf6-f3f5aa6f3145`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:31:51,361::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:31:51,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`397131a6-4772-4c0f-bbe3-bf03d157f541`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:31:51,362::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:31:51,362::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c8330cff-74eb-4f51-85a0-464664c02151`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:31:51,362::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:31:51,362::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:31:51,362::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c8330cff-74eb-4f51-85a0-464664c02151`::Granted request >Thread-46::DEBUG::2014-08-22 09:31:51,362::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`397131a6-4772-4c0f-bbe3-bf03d157f541`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:31:51,363::task::993::Storage.TaskManager.Task::(_decref) Task=`397131a6-4772-4c0f-bbe3-bf03d157f541`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:31:51,364::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:51,365::task::1191::Storage.TaskManager.Task::(prepare) Task=`397131a6-4772-4c0f-bbe3-bf03d157f541`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:31:51,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`397131a6-4772-4c0f-bbe3-bf03d157f541`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:31:51,365::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:31:51,365::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:31:51,365::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:31:51,365::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:31:51,365::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:31:51,365::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:31:51,365::task::993::Storage.TaskManager.Task::(_decref) Task=`397131a6-4772-4c0f-bbe3-bf03d157f541`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:31:52,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`88de758f-dab0-4285-a754-1e33c958c8e0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:31:52,227::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:31:52,228::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000166207', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000230101', 'lastCheck': '4.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:52,228::task::1191::Storage.TaskManager.Task::(prepare) Task=`88de758f-dab0-4285-a754-1e33c958c8e0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000166207', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000230101', 'lastCheck': '4.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:31:52,228::task::595::Storage.TaskManager.Task::(_updateState) Task=`88de758f-dab0-4285-a754-1e33c958c8e0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:31:52,228::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:31:52,228::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:31:52,228::task::993::Storage.TaskManager.Task::(_decref) Task=`88de758f-dab0-4285-a754-1e33c958c8e0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:31:52,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:52,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011561 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:54,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:54,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133395 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:31:56,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:56,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116702 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:57,177::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,177::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:57,177::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,178::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:31:57,179::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:57,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,185::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:31:57,186::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000228433 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:57,186::__init__::225::IOProcess::(_processLogs) (696) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) (696) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) (696) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) (696) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:31:57,187::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) (697) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) (697) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:57,188::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|(697) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,189::__init__::225::IOProcess::(_processLogs) (697) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,189::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,189::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:57,592::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,592::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:57,592::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,593::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,593::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:31:57,593::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:31:57,594::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:57,600::__init__::225::IOProcess::(_processLogs) (698) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:31:57,600::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206808 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:31:57,601::__init__::225::IOProcess::(_processLogs) (698) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) (698) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) (698) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,602::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,603::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:57,603::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,603::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:57,603::__init__::225::IOProcess::(_processLogs) (699) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:31:57,603::__init__::225::IOProcess::(_processLogs) (699) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:31:57,604::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|(699) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,604::__init__::225::IOProcess::(_processLogs) (699) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,604::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,604::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:31:57,604::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,605::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:31:57,605::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,605::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,605::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:57,605::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,605::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) (700) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) (700) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) (700) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) (700) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:31:57,606::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) (701) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) (701) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) (701) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) (701) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,607::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,608::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:31:57,608::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:31:57,608::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:31:57,608::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:31:57,608::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:31:57,608::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) (702) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) (702) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) (702) Building response >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) (702) Queuing response >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:31:57,609::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:31:58,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:31:58,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119259 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:00,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:00,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116634 s, 87.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:01,419::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:32:01,419::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d044195-8fba-42e7-a406-1d9ca21dc871`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:01,419::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:32:01,420::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:01,420::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d044195-8fba-42e7-a406-1d9ca21dc871`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:01,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d044195-8fba-42e7-a406-1d9ca21dc871`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:01,420::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:01,420::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:01,420::task::993::Storage.TaskManager.Task::(_decref) Task=`8d044195-8fba-42e7-a406-1d9ca21dc871`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:32:01,476::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:32:01,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`5459956a-accb-4500-9e21-ebb98f479c8a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:32:01,476::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:32:01,477::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`18c2d319-2d42-4994-b5b7-6d8b830fe5eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:32:01,477::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:32:01,477::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:32:01,477::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`18c2d319-2d42-4994-b5b7-6d8b830fe5eb`::Granted request >Thread-46::DEBUG::2014-08-22 09:32:01,477::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5459956a-accb-4500-9e21-ebb98f479c8a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:32:01,477::task::993::Storage.TaskManager.Task::(_decref) Task=`5459956a-accb-4500-9e21-ebb98f479c8a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:32:01,479::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:01,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`5459956a-accb-4500-9e21-ebb98f479c8a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:01,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`5459956a-accb-4500-9e21-ebb98f479c8a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:32:01,480::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:32:01,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:32:01,480::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:32:01,480::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:32:01,480::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:32:01,480::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:32:01,480::task::993::Storage.TaskManager.Task::(_decref) Task=`5459956a-accb-4500-9e21-ebb98f479c8a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:02,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:02,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208494 s, 49.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:04,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:05,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118229 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:07,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:07,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115821 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:07,190::__init__::225::IOProcess::(_processLogs) (690) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:07,190::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:07,190::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:07,190::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:07,191::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:32:07,191::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:07,191::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,197::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,198::__init__::225::IOProcess::(_processLogs) (703) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:32:07,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000252683 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:07,199::__init__::225::IOProcess::(_processLogs) (703) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:07,199::__init__::225::IOProcess::(_processLogs) (703) Building response >Thread-17::DEBUG::2014-08-22 09:32:07,199::__init__::225::IOProcess::(_processLogs) (703) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,199::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,199::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:07,199::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:07,200::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:07,200::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:07,200::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,200::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,200::__init__::225::IOProcess::(_processLogs) (704) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:07,200::__init__::225::IOProcess::(_processLogs) (704) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:07,201::__init__::225::IOProcess::(_processLogs) (704) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,201::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,201::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-47::DEBUG::2014-08-22 09:32:07,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`174f1741-5d13-4e19-96dc-8075e54cceab`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:07,511::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:32:07,511::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252683', 'lastCheck': '0.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206808', 'lastCheck': '9.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:07,511::task::1191::Storage.TaskManager.Task::(prepare) Task=`174f1741-5d13-4e19-96dc-8075e54cceab`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252683', 'lastCheck': '0.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206808', 'lastCheck': '9.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:07,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`174f1741-5d13-4e19-96dc-8075e54cceab`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:07,511::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:07,511::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:07,511::task::993::Storage.TaskManager.Task::(_decref) Task=`174f1741-5d13-4e19-96dc-8075e54cceab`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:32:07,609::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:07,609::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:07,609::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:07,609::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,610::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,610::__init__::225::IOProcess::(_processLogs) (705) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:07,610::__init__::225::IOProcess::(_processLogs) (705) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:07,610::__init__::225::IOProcess::(_processLogs) (705) Building response >Thread-17::DEBUG::2014-08-22 09:32:07,610::__init__::225::IOProcess::(_processLogs) (705) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,610::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,611::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:32:07,611::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:32:07,617::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000316759 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:07,618::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:07,618::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:07,618::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:07,618::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,618::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,618::__init__::225::IOProcess::(_processLogs) (706) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:07,619::__init__::225::IOProcess::(_processLogs) (706) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:07,619::__init__::225::IOProcess::(_processLogs) (706) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,620::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,620::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:07,620::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:32:07,620::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) (707) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) (707) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) (707) Building response >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) (707) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,621::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:07,622::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:32:07,622::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:07,622::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:07,622::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,622::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,622::__init__::225::IOProcess::(_processLogs) (708) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) (708) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) (708) Building response >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) (708) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:32:07,623::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:07,624::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:07,624::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:07,624::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:07,624::__init__::225::IOProcess::(_processLogs) (709) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:07,624::__init__::225::IOProcess::(_processLogs) (709) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:07,624::__init__::225::IOProcess::(_processLogs) (709) Building response >Thread-17::DEBUG::2014-08-22 09:32:07,625::__init__::225::IOProcess::(_processLogs) (709) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:07,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:07,625::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:32:09,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:09,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121531 s, 84.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:11,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:11,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116054 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:11,523::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:32:11,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fc0588a-6c80-4bed-b359-19a5d4e65f5c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:11,523::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:32:11,524::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:11,524::task::1191::Storage.TaskManager.Task::(prepare) Task=`7fc0588a-6c80-4bed-b359-19a5d4e65f5c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:11,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fc0588a-6c80-4bed-b359-19a5d4e65f5c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:11,524::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:11,524::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:11,524::task::993::Storage.TaskManager.Task::(_decref) Task=`7fc0588a-6c80-4bed-b359-19a5d4e65f5c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:32:11,549::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:32:11,549::task::595::Storage.TaskManager.Task::(_updateState) Task=`700a19bc-3f51-4fa0-8f19-c42b353e497c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:32:11,549::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:32:11,550::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cd7abb56-1d53-4203-9070-ecdbf7a6817b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:32:11,550::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:32:11,550::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:32:11,550::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cd7abb56-1d53-4203-9070-ecdbf7a6817b`::Granted request >Thread-46::DEBUG::2014-08-22 09:32:11,550::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`700a19bc-3f51-4fa0-8f19-c42b353e497c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:32:11,550::task::993::Storage.TaskManager.Task::(_decref) Task=`700a19bc-3f51-4fa0-8f19-c42b353e497c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:32:11,552::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:11,552::task::1191::Storage.TaskManager.Task::(prepare) Task=`700a19bc-3f51-4fa0-8f19-c42b353e497c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:11,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`700a19bc-3f51-4fa0-8f19-c42b353e497c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:32:11,552::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:32:11,553::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:32:11,553::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:32:11,553::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:32:11,553::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:32:11,553::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:32:11,553::task::993::Storage.TaskManager.Task::(_decref) Task=`700a19bc-3f51-4fa0-8f19-c42b353e497c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:13,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:13,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115894 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:15,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:15,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133699 s, 76.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) (710) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:17,202::__init__::225::IOProcess::(_processLogs) (710) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:17,203::__init__::225::IOProcess::(_processLogs) (710) Queuing response >Thread-21::DEBUG::2014-08-22 09:32:17,204::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:17,204::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,210::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:32:17,211::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00022244 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:17,211::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) (711) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:17,212::__init__::225::IOProcess::(_processLogs) (711) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:17,213::__init__::225::IOProcess::(_processLogs) DEBUG|(711) Building response >Thread-17::DEBUG::2014-08-22 09:32:17,214::__init__::225::IOProcess::(_processLogs) (711) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:17,214::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,214::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:32:17,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:17,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117312 s, 87.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,625::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,626::__init__::225::IOProcess::(_processLogs) (712) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:17,626::__init__::225::IOProcess::(_processLogs) (712) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:17,626::__init__::225::IOProcess::(_processLogs) DEBUG|(712) Building response >Thread-17::DEBUG::2014-08-22 09:32:17,627::__init__::225::IOProcess::(_processLogs) (712) Queuing response >Thread-22::DEBUG::2014-08-22 09:32:17,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:17,627::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,633::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:32:17,634::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000188208 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) (713) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:17,635::__init__::225::IOProcess::(_processLogs) (713) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:17,636::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(713) Building response >Thread-17::DEBUG::2014-08-22 09:32:17,637::__init__::225::IOProcess::(_processLogs) (713) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:17,637::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,637::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:17,637::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:17,637::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:32:17,637::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:17,638::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,638::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:17,638::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,638::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,638::__init__::225::IOProcess::(_processLogs) (714) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:17,638::__init__::225::IOProcess::(_processLogs) (714) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) (714) Building response >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) (714) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:32:17,639::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) (715) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) (715) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) (715) Building response >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) (715) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:17,640::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,641::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:17,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:17,641::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:32:17,641::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:17,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:17,642::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:17,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:17,642::__init__::225::IOProcess::(_processLogs) (716) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:17,642::__init__::225::IOProcess::(_processLogs) (716) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:17,642::__init__::225::IOProcess::(_processLogs) (716) Building response >Thread-17::DEBUG::2014-08-22 09:32:17,642::__init__::225::IOProcess::(_processLogs) (716) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:17,643::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:17,643::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:32:19,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:19,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119977 s, 85.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:21,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:21,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121161 s, 84.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:21,602::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:32:21,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`65695fa5-5245-4254-b872-aeea23336ebd`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:21,603::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:32:21,603::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:21,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`65695fa5-5245-4254-b872-aeea23336ebd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:21,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`65695fa5-5245-4254-b872-aeea23336ebd`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:21,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:21,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:21,603::task::993::Storage.TaskManager.Task::(_decref) Task=`65695fa5-5245-4254-b872-aeea23336ebd`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:32:21,628::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:32:21,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0100bf2-bbb8-49f1-bbf1-799e5006ec84`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:32:21,628::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:32:21,629::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`07729c9c-d1a9-451f-94af-c64e026565ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:32:21,629::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:32:21,629::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:32:21,629::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`07729c9c-d1a9-451f-94af-c64e026565ca`::Granted request >Thread-46::DEBUG::2014-08-22 09:32:21,629::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f0100bf2-bbb8-49f1-bbf1-799e5006ec84`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:32:21,629::task::993::Storage.TaskManager.Task::(_decref) Task=`f0100bf2-bbb8-49f1-bbf1-799e5006ec84`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:32:21,631::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:21,631::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0100bf2-bbb8-49f1-bbf1-799e5006ec84`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:21,631::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0100bf2-bbb8-49f1-bbf1-799e5006ec84`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:32:21,632::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:32:21,632::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:32:21,632::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:32:21,632::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:32:21,632::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:32:21,632::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:32:21,632::task::993::Storage.TaskManager.Task::(_decref) Task=`f0100bf2-bbb8-49f1-bbf1-799e5006ec84`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:32:22,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`abfb455b-4396-4e0e-a16e-a99455e98456`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:22,714::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:32:22,714::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00022244', 'lastCheck': '5.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000188208', 'lastCheck': '5.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:22,714::task::1191::Storage.TaskManager.Task::(prepare) Task=`abfb455b-4396-4e0e-a16e-a99455e98456`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00022244', 'lastCheck': '5.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000188208', 'lastCheck': '5.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:22,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`abfb455b-4396-4e0e-a16e-a99455e98456`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:22,714::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:22,714::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:22,714::task::993::Storage.TaskManager.Task::(_decref) Task=`abfb455b-4396-4e0e-a16e-a99455e98456`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:23,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:23,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121631 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:25,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:25,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155526 s, 65.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:27,214::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:27,214::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:27,214::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:27,214::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,214::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,214::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,215::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,215::__init__::225::IOProcess::(_processLogs) (717) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:27,215::__init__::225::IOProcess::(_processLogs) (717) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:27,215::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(717) Building response >Thread-17::DEBUG::2014-08-22 09:32:27,216::__init__::225::IOProcess::(_processLogs) (717) Queuing response >Thread-21::DEBUG::2014-08-22 09:32:27,216::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:27,217::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,222::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:32:27,223::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000173437 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:27,224::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:27,224::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:27,224::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:27,225::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,225::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) (718) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) (718) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) (718) Building response >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) (718) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,226::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:32:27,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:27,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120034 s, 85.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) (719) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:27,643::__init__::225::IOProcess::(_processLogs) (719) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:27,644::__init__::225::IOProcess::(_processLogs) (719) Queuing response >Thread-22::DEBUG::2014-08-22 09:32:27,645::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:27,645::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,650::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:32:27,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000266368 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) (720) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:27,652::__init__::225::IOProcess::(_processLogs) (720) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:27,653::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(720) Building response >Thread-17::DEBUG::2014-08-22 09:32:27,654::__init__::225::IOProcess::(_processLogs) (720) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:27,654::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,654::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:27,654::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:27,654::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:32:27,654::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) (721) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) (721) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) (721) Building response >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) (721) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:27,655::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,656::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:27,656::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:27,656::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:32:27,656::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:27,656::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) (722) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) (722) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) (722) Building response >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) (722) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:27,657::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:27,658::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:32:27,658::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:27,658::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:27,658::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) (723) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) (723) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) (723) Building response >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) (723) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:27,659::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:32:29,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:29,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118178 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:31,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:31,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119724 s, 85.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:31,681::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:32:31,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`1da36dda-72a4-4162-9ea2-b151fe22e838`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:31,682::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:32:31,682::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:31,682::task::1191::Storage.TaskManager.Task::(prepare) Task=`1da36dda-72a4-4162-9ea2-b151fe22e838`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:31,682::task::595::Storage.TaskManager.Task::(_updateState) Task=`1da36dda-72a4-4162-9ea2-b151fe22e838`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:31,682::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:31,682::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:31,682::task::993::Storage.TaskManager.Task::(_decref) Task=`1da36dda-72a4-4162-9ea2-b151fe22e838`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:32:31,715::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:32:31,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`1828d2d0-1d87-4b12-aedf-f18ecd0e3afb`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:32:31,715::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:32:31,716::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0fc9ade9-ddbf-4e20-9afa-a00de5904cca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:32:31,716::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:32:31,716::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:32:31,716::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0fc9ade9-ddbf-4e20-9afa-a00de5904cca`::Granted request >Thread-46::DEBUG::2014-08-22 09:32:31,716::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1828d2d0-1d87-4b12-aedf-f18ecd0e3afb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:32:31,716::task::993::Storage.TaskManager.Task::(_decref) Task=`1828d2d0-1d87-4b12-aedf-f18ecd0e3afb`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:32:31,718::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:31,718::task::1191::Storage.TaskManager.Task::(prepare) Task=`1828d2d0-1d87-4b12-aedf-f18ecd0e3afb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:31,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`1828d2d0-1d87-4b12-aedf-f18ecd0e3afb`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:32:31,719::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:32:31,719::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:32:31,719::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:32:31,719::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:32:31,719::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:32:31,719::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:32:31,719::task::993::Storage.TaskManager.Task::(_decref) Task=`1828d2d0-1d87-4b12-aedf-f18ecd0e3afb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:33,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:33,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116781 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:35,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:35,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118049 s, 86.7 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:32:37,225::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:32:37,227::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,227::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:32:37,227::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) (724) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) (724) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) (724) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,228::__init__::225::IOProcess::(_processLogs) (724) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,229::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:32:37,230::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,230::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:32:37,231::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,231::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,232::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,233::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,233::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,234::__init__::225::IOProcess::(_processLogs) (725) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:32:37,234::__init__::225::IOProcess::(_processLogs) (725) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:32:37,236::__init__::225::IOProcess::(_processLogs) (725) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,237::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,237::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-21::DEBUG::2014-08-22 09:32:37,238::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:32:37,238::__init__::225::IOProcess::(_processLogs) Message size is 169 >Thread-17::DEBUG::2014-08-22 09:32:37,239::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,239::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,239::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,239::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,239::__init__::225::IOProcess::(_processLogs) (726) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:32:37,239::__init__::225::IOProcess::(_processLogs) (726) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:32:37,259::__init__::225::IOProcess::(_processLogs) (726) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,259::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-21::DEBUG::2014-08-22 09:32:37,259::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:32:37,260::__init__::225::IOProcess::(_processLogs) Message size is 193 >Thread-17::DEBUG::2014-08-22 09:32:37,260::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,260::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,260::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,261::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,261::__init__::225::IOProcess::(_processLogs) (727) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:32:37,261::__init__::225::IOProcess::(_processLogs) (727) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:32:37,263::__init__::225::IOProcess::(_processLogs) (727) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,263::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:32:37,263::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:32:37,264::__init__::225::IOProcess::(_processLogs) Sending response sized 603 >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,265::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) (728) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) (728) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) (728) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) (728) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,266::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,267::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,267::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,267::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,267::__init__::225::IOProcess::(_processLogs) (729) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:32:37,267::__init__::225::IOProcess::(_processLogs) (729) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 09:32:37,267::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:32:37,267::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(729) Building response >Thread-21::INFO::2014-08-22 09:32:37,268::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:32:37,268::__init__::225::IOProcess::(_processLogs) (729) Queuing response >Thread-21::INFO::2014-08-22 09:32:37,268::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:32:37,268::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,269::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,270::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,270::__init__::225::IOProcess::(_processLogs) (730) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:37,270::__init__::225::IOProcess::(_processLogs) (730) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:37,270::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(730) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,271::__init__::225::IOProcess::(_processLogs) (730) Queuing response >Thread-21::DEBUG::2014-08-22 09:32:37,271::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:37,271::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,277::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:32:37,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000161003 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,279::__init__::225::IOProcess::(_processLogs) (731) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:37,280::__init__::225::IOProcess::(_processLogs) (731) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:37,280::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(731) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,281::__init__::225::IOProcess::(_processLogs) (731) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,281::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,281::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:32:37,659::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:32:37,660::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,660::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:32:37,661::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,661::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,661::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) (732) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) (732) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) (732) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) (732) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:32:37,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) (733) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) (733) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:32:37,663::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(733) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,664::__init__::225::IOProcess::(_processLogs) (733) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,665::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,665::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-22::DEBUG::2014-08-22 09:32:37,667::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:32:37,667::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,667::__init__::225::IOProcess::(_processLogs) Message size is 171 >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) (734) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:32:37,668::__init__::225::IOProcess::(_processLogs) (734) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:32:37,727::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(734) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,727::__init__::225::IOProcess::(_processLogs) (734) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,727::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:32:37,728::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:32:37,728::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:37,728::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) Message size is 195 >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) (735) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:32:37,729::__init__::225::IOProcess::(_processLogs) (735) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:32:37,730::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(735) Building response >Thread-22::DEBUG::2014-08-22 09:32:37,730::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:32:37,730::__init__::225::IOProcess::(_processLogs) (735) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,731::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,731::__init__::225::IOProcess::(_processLogs) Sending response sized 831 >Thread-17::DEBUG::2014-08-22 09:32:37,731::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,731::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) (736) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) (736) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:37,732::__init__::225::IOProcess::(_processLogs) (736) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) (736) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,733::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,734::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,734::__init__::225::IOProcess::(_processLogs) (737) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:32:37,734::__init__::225::IOProcess::(_processLogs) (737) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:32:37,734::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(737) Building response >Thread-22::DEBUG::2014-08-22 09:32:37,734::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:32:37,735::__init__::225::IOProcess::(_processLogs) (737) Queuing response >Thread-22::INFO::2014-08-22 09:32:37,735::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:32:37,735::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 09:32:37,735::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:32:37,736::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-17::DEBUG::2014-08-22 09:32:37,736::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,736::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:37,736::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,737::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:32:37,737::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:37,737::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,743::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:32:37,744::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000237997 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:37,744::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,745::__init__::225::IOProcess::(_processLogs) (738) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:37,745::__init__::225::IOProcess::(_processLogs) (738) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:37,745::__init__::225::IOProcess::(_processLogs) (738) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,745::__init__::225::IOProcess::(_processLogs) (738) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,745::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,746::__init__::225::IOProcess::(_processLogs) (739) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:37,747::__init__::225::IOProcess::(_processLogs) (739) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:37,747::__init__::225::IOProcess::(_processLogs) (739) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,748::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,748::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:37,748::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:32:37,748::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) (740) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) (740) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) (740) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) (740) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,749::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,750::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:37,750::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:32:37,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,750::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,750::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) (741) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) (741) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) (741) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) (741) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:32:37,751::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:37,752::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:37,752::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:37,752::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:37,753::__init__::225::IOProcess::(_processLogs) (742) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:37,753::__init__::225::IOProcess::(_processLogs) (742) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:37,753::__init__::225::IOProcess::(_processLogs) (742) Building response >Thread-17::DEBUG::2014-08-22 09:32:37,753::__init__::225::IOProcess::(_processLogs) (742) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:37,753::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:37,753::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:32:37,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:37,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115439 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:37,899::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff5bcac9-7e24-436b-a8c3-71bc91a2add9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:37,899::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:32:37,899::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000161003', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237997', 'lastCheck': '0.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:37,900::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff5bcac9-7e24-436b-a8c3-71bc91a2add9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000161003', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237997', 'lastCheck': '0.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:37,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff5bcac9-7e24-436b-a8c3-71bc91a2add9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:37,900::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:37,900::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:37,900::task::993::Storage.TaskManager.Task::(_decref) Task=`ff5bcac9-7e24-436b-a8c3-71bc91a2add9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:39,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:39,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122583 s, 83.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:41,768::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:32:41,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`fafa3799-5858-432a-89ba-5efa0c47e37d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:41,769::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:32:41,769::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:41,769::task::1191::Storage.TaskManager.Task::(prepare) Task=`fafa3799-5858-432a-89ba-5efa0c47e37d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:41,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`fafa3799-5858-432a-89ba-5efa0c47e37d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:41,769::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:41,769::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:41,769::task::993::Storage.TaskManager.Task::(_decref) Task=`fafa3799-5858-432a-89ba-5efa0c47e37d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:32:41,807::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:32:41,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ff1c69e-3365-4ecd-bf3f-3acad1c6c79a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:32:41,807::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:32:41,807::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`18083360-d455-4ba7-a1d6-3a6d18b7e98c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:32:41,808::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:32:41,808::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:32:41,808::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`18083360-d455-4ba7-a1d6-3a6d18b7e98c`::Granted request >Thread-46::DEBUG::2014-08-22 09:32:41,808::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7ff1c69e-3365-4ecd-bf3f-3acad1c6c79a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:32:41,808::task::993::Storage.TaskManager.Task::(_decref) Task=`7ff1c69e-3365-4ecd-bf3f-3acad1c6c79a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:32:41,810::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:41,810::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ff1c69e-3365-4ecd-bf3f-3acad1c6c79a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:41,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ff1c69e-3365-4ecd-bf3f-3acad1c6c79a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:32:41,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:32:41,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:32:41,811::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:32:41,811::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:32:41,811::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:32:41,811::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:32:41,811::task::993::Storage.TaskManager.Task::(_decref) Task=`7ff1c69e-3365-4ecd-bf3f-3acad1c6c79a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:41,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:41,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117054 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:43,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:43,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0150494 s, 68.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:45,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:46,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119755 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:47,281::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:47,282::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:47,282::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,282::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,282::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,282::__init__::225::IOProcess::(_processLogs) (743) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:47,282::__init__::225::IOProcess::(_processLogs) (743) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:47,283::__init__::225::IOProcess::(_processLogs) (743) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,283::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:32:47,283::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:47,284::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:32:47,290::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000200131 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) (744) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:47,291::__init__::225::IOProcess::(_processLogs) (744) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:47,292::__init__::225::IOProcess::(_processLogs) (744) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,293::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:47,293::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) (745) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:47,753::__init__::225::IOProcess::(_processLogs) (745) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:47,754::__init__::225::IOProcess::(_processLogs) (745) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,755::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:32:47,755::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:32:47,761::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216314 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) (746) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:47,762::__init__::225::IOProcess::(_processLogs) (746) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:47,764::__init__::225::IOProcess::(_processLogs) (746) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,765::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:47,765::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:47,766::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:32:47,766::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:47,766::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,766::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,766::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,766::__init__::225::IOProcess::(_processLogs) (747) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) (747) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) (747) Building response >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) (747) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:32:47,767::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) (748) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) (748) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) (748) Building response >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) (748) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:47,768::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:47,769::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:32:47,769::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:47,769::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:47,769::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) (749) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) (749) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) (749) Building response >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) (749) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:47,770::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:32:48,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:48,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120583 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:50,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:50,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117432 s, 87.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:51,869::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:32:51,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f6f664b-16fe-436f-abaa-edd912b83287`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:51,869::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:32:51,869::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:51,869::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f6f664b-16fe-436f-abaa-edd912b83287`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:32:51,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f6f664b-16fe-436f-abaa-edd912b83287`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:51,870::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:51,870::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:51,870::task::993::Storage.TaskManager.Task::(_decref) Task=`1f6f664b-16fe-436f-abaa-edd912b83287`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:32:51,902::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:32:51,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd11e7d7-1068-494d-a87f-f67264d46a19`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:32:51,903::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:32:51,903::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4d4b1f3b-e3d6-45e2-a50f-b87dcadd4167`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:32:51,903::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:32:51,903::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:32:51,903::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4d4b1f3b-e3d6-45e2-a50f-b87dcadd4167`::Granted request >Thread-46::DEBUG::2014-08-22 09:32:51,903::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dd11e7d7-1068-494d-a87f-f67264d46a19`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:32:51,904::task::993::Storage.TaskManager.Task::(_decref) Task=`dd11e7d7-1068-494d-a87f-f67264d46a19`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:32:51,905::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:51,906::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd11e7d7-1068-494d-a87f-f67264d46a19`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:32:51,906::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd11e7d7-1068-494d-a87f-f67264d46a19`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:32:51,906::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:32:51,906::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:32:51,906::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:32:51,906::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:32:51,906::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:32:51,906::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:32:51,906::task::993::Storage.TaskManager.Task::(_decref) Task=`dd11e7d7-1068-494d-a87f-f67264d46a19`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:52,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:52,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119954 s, 85.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:32:53,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a8c3735-0102-43bd-93c9-be5a0d5387a9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:32:53,088::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:32:53,088::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000200131', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216314', 'lastCheck': '5.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:53,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`9a8c3735-0102-43bd-93c9-be5a0d5387a9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000200131', 'lastCheck': '5.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216314', 'lastCheck': '5.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:32:53,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a8c3735-0102-43bd-93c9-be5a0d5387a9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:32:53,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:32:53,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:32:53,089::task::993::Storage.TaskManager.Task::(_decref) Task=`9a8c3735-0102-43bd-93c9-be5a0d5387a9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:32:54,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:54,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115897 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:32:56,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:56,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226339 s, 45.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:57,293::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:57,293::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,293::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,294::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,294::__init__::225::IOProcess::(_processLogs) (750) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:57,294::__init__::225::IOProcess::(_processLogs) (750) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:57,294::__init__::225::IOProcess::(_processLogs) (750) Queuing response >Thread-21::DEBUG::2014-08-22 09:32:57,295::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:57,295::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,301::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:32:57,302::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000197848 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) (751) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:57,303::__init__::225::IOProcess::(_processLogs) (751) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:57,304::__init__::225::IOProcess::(_processLogs) (751) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:57,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,305::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:57,770::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:57,770::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:57,770::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,770::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,770::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,771::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,771::__init__::225::IOProcess::(_processLogs) (752) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:57,771::__init__::225::IOProcess::(_processLogs) (752) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:57,771::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(752) Building response >Thread-17::DEBUG::2014-08-22 09:32:57,772::__init__::225::IOProcess::(_processLogs) (752) Queuing response >Thread-22::DEBUG::2014-08-22 09:32:57,772::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:32:57,773::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,779::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:32:57,779::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000253263 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) (753) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:32:57,780::__init__::225::IOProcess::(_processLogs) (753) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:32:57,781::__init__::225::IOProcess::(_processLogs) (753) Building response >Thread-17::DEBUG::2014-08-22 09:32:57,781::__init__::225::IOProcess::(_processLogs) (753) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:57,781::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,781::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:32:57,782::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:57,782::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:32:57,782::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) (754) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) (754) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:57,783::__init__::225::IOProcess::(_processLogs) (754) Building response >Thread-17::DEBUG::2014-08-22 09:32:57,784::__init__::225::IOProcess::(_processLogs) (754) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:57,784::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,784::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:57,784::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:57,784::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:32:57,784::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) (755) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) (755) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) (755) Building response >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) (755) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:57,785::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,786::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:32:57,786::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:32:57,786::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:32:57,786::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:32:57,786::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:32:57,786::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:32:57,787::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:32:57,787::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:32:57,787::__init__::225::IOProcess::(_processLogs) (756) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:32:57,787::__init__::225::IOProcess::(_processLogs) (756) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:32:57,787::__init__::225::IOProcess::(_processLogs) (756) Building response >Thread-17::DEBUG::2014-08-22 09:32:57,787::__init__::225::IOProcess::(_processLogs) (756) Queuing response >Thread-17::DEBUG::2014-08-22 09:32:57,788::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:32:57,788::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:32:58,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:32:58,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117439 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:00,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:00,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121547 s, 84.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:01,992::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:33:01,994::task::595::Storage.TaskManager.Task::(_updateState) Task=`7894da00-1a51-415c-94c5-c26f7e25ff66`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:01,994::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:33:01,996::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:01,997::task::1191::Storage.TaskManager.Task::(prepare) Task=`7894da00-1a51-415c-94c5-c26f7e25ff66`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:01,997::task::595::Storage.TaskManager.Task::(_updateState) Task=`7894da00-1a51-415c-94c5-c26f7e25ff66`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:01,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:01,999::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:01,999::task::993::Storage.TaskManager.Task::(_decref) Task=`7894da00-1a51-415c-94c5-c26f7e25ff66`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:33:02,280::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:33:02,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab208d3e-aa5c-404c-b7bb-4d5c11daebff`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:33:02,282::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:33:02,283::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d8694470-697c-47e7-a8c1-ef2088af6e2b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:33:02,283::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:33:02,284::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:33:02,284::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d8694470-697c-47e7-a8c1-ef2088af6e2b`::Granted request >Thread-46::DEBUG::2014-08-22 09:33:02,285::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ab208d3e-aa5c-404c-b7bb-4d5c11daebff`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:33:02,286::task::993::Storage.TaskManager.Task::(_decref) Task=`ab208d3e-aa5c-404c-b7bb-4d5c11daebff`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:33:02,295::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:02,295::task::1191::Storage.TaskManager.Task::(prepare) Task=`ab208d3e-aa5c-404c-b7bb-4d5c11daebff`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:02,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab208d3e-aa5c-404c-b7bb-4d5c11daebff`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:33:02,296::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:33:02,296::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:33:02,297::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:33:02,298::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:33:02,298::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:33:02,298::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:33:02,299::task::993::Storage.TaskManager.Task::(_decref) Task=`ab208d3e-aa5c-404c-b7bb-4d5c11daebff`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:02,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:02,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209863 s, 48.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:04,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:04,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117111 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:06,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:06,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011774 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) (757) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:07,306::__init__::225::IOProcess::(_processLogs) (757) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:07,307::__init__::225::IOProcess::(_processLogs) DEBUG|(757) Building response >Thread-21::DEBUG::2014-08-22 09:33:07,307::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:07,308::__init__::225::IOProcess::(_processLogs) (757) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:07,314::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:33:07,314::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000147538 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:07,316::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,316::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:07,316::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,316::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,316::__init__::225::IOProcess::(_processLogs) DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) (758) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) (758) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) (758) Building response >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) (758) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:07,318::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:07,787::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,787::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) (759) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) (759) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) (759) Building response >Thread-17::DEBUG::2014-08-22 09:33:07,788::__init__::225::IOProcess::(_processLogs) (759) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,789::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:33:07,789::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:07,790::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:33:07,796::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000212416 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:07,796::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,796::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:07,796::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,796::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,797::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,797::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,797::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,797::__init__::225::IOProcess::(_processLogs) (760) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:07,797::__init__::225::IOProcess::(_processLogs) (760) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:07,797::__init__::225::IOProcess::(_processLogs) DEBUG|(760) Building response >Thread-17::DEBUG::2014-08-22 09:33:07,798::__init__::225::IOProcess::(_processLogs) (760) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,798::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:07,798::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:07,798::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,799::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:33:07,799::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,799::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,799::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,799::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) (761) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) (761) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) (761) Building response >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) (761) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:33:07,800::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) (762) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) (762) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:07,801::__init__::225::IOProcess::(_processLogs) (762) Building response >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) (762) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:07,802::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:07,803::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:07,803::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:07,803::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:07,803::__init__::225::IOProcess::(_processLogs) (763) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:07,803::__init__::225::IOProcess::(_processLogs) (763) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:07,803::__init__::225::IOProcess::(_processLogs) (763) Building response >Thread-17::DEBUG::2014-08-22 09:33:07,804::__init__::225::IOProcess::(_processLogs) (763) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:07,804::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:07,804::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:33:08,273::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca184bb6-e7b3-4654-961a-f133835225fb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:08,273::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:33:08,273::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000147538', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000212416', 'lastCheck': '0.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:08,274::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca184bb6-e7b3-4654-961a-f133835225fb`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000147538', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000212416', 'lastCheck': '0.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:08,274::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca184bb6-e7b3-4654-961a-f133835225fb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:08,274::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:08,274::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:08,274::task::993::Storage.TaskManager.Task::(_decref) Task=`ca184bb6-e7b3-4654-961a-f133835225fb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:08,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:08,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120724 s, 84.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:10,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:10,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143687 s, 71.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:12,350::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [2db01973] >Thread-47::DEBUG::2014-08-22 09:33:12,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bc20dda-7a05-4c97-af83-cd89531cf154`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:12,351::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:33:12,351::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:12,351::task::1191::Storage.TaskManager.Task::(prepare) Task=`0bc20dda-7a05-4c97-af83-cd89531cf154`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:12,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bc20dda-7a05-4c97-af83-cd89531cf154`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:12,351::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:12,351::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:12,352::task::993::Storage.TaskManager.Task::(_decref) Task=`0bc20dda-7a05-4c97-af83-cd89531cf154`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:33:12,385::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [2db01973] >Thread-46::DEBUG::2014-08-22 09:33:12,386::task::595::Storage.TaskManager.Task::(_updateState) Task=`f697b29e-3f92-4b82-a878-c8471f5dd926`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:33:12,386::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:33:12,386::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`dc81b891-37c7-477e-8890-1d7a827910ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:33:12,386::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:33:12,386::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:33:12,386::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`dc81b891-37c7-477e-8890-1d7a827910ec`::Granted request >Thread-46::DEBUG::2014-08-22 09:33:12,386::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f697b29e-3f92-4b82-a878-c8471f5dd926`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:33:12,387::task::993::Storage.TaskManager.Task::(_decref) Task=`f697b29e-3f92-4b82-a878-c8471f5dd926`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:33:12,388::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:12,389::task::1191::Storage.TaskManager.Task::(prepare) Task=`f697b29e-3f92-4b82-a878-c8471f5dd926`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:12,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`f697b29e-3f92-4b82-a878-c8471f5dd926`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:33:12,389::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:33:12,389::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:33:12,389::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:33:12,389::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:33:12,389::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:33:12,389::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:33:12,389::task::993::Storage.TaskManager.Task::(_decref) Task=`f697b29e-3f92-4b82-a878-c8471f5dd926`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:12,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:12,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115492 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:14,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:15,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137899 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:17,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:17,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118203 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:17,318::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,318::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:17,318::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,318::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,318::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,318::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,319::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,319::__init__::225::IOProcess::(_processLogs) (764) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:17,319::__init__::225::IOProcess::(_processLogs) (764) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:17,319::__init__::225::IOProcess::(_processLogs) DEBUG|(764) Building response >Thread-17::DEBUG::2014-08-22 09:33:17,320::__init__::225::IOProcess::(_processLogs) (764) Queuing response >Thread-21::DEBUG::2014-08-22 09:33:17,320::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:17,321::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,326::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:33:17,327::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000146485 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,328::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,329::__init__::225::IOProcess::(_processLogs) (765) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:17,329::__init__::225::IOProcess::(_processLogs) (765) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:17,329::__init__::225::IOProcess::(_processLogs) DEBUG|(765) Building response >Thread-17::DEBUG::2014-08-22 09:33:17,330::__init__::225::IOProcess::(_processLogs) (765) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:17,330::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,330::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:17,803::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) (766) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:17,804::__init__::225::IOProcess::(_processLogs) (766) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:17,805::__init__::225::IOProcess::(_processLogs) DEBUG|(766) Building response >Thread-22::DEBUG::2014-08-22 09:33:17,806::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:17,806::__init__::225::IOProcess::(_processLogs) (766) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:17,812::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,812::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:33:17,812::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216411 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:17,813::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) (767) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:17,814::__init__::225::IOProcess::(_processLogs) (767) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:17,815::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(767) Building response >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) (767) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,816::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,817::__init__::225::IOProcess::(_processLogs) (768) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:17,817::__init__::225::IOProcess::(_processLogs) (768) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:17,817::__init__::225::IOProcess::(_processLogs) (768) Building response >Thread-17::DEBUG::2014-08-22 09:33:17,817::__init__::225::IOProcess::(_processLogs) (768) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:17,817::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,817::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:17,818::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,818::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:33:17,818::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,818::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,818::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) (769) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) (769) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) (769) Building response >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) (769) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:17,819::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:33:17,820::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:17,820::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:17,820::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) (770) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) (770) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) (770) Building response >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) (770) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:17,821::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:33:19,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:19,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012153 s, 84.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:21,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:21,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017743 s, 57.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:22,439::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:33:22,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`b284ab9e-915f-49ce-9c0b-59cc2134fe18`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:22,440::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:33:22,440::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:22,440::task::1191::Storage.TaskManager.Task::(prepare) Task=`b284ab9e-915f-49ce-9c0b-59cc2134fe18`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:22,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`b284ab9e-915f-49ce-9c0b-59cc2134fe18`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:22,440::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:22,440::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:22,440::task::993::Storage.TaskManager.Task::(_decref) Task=`b284ab9e-915f-49ce-9c0b-59cc2134fe18`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:33:22,573::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:33:22,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`50a09a17-0453-488a-8bb6-cfd22265a378`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:33:22,573::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:33:22,573::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e071e4cf-68b5-49f3-83f8-b45307a32139`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:33:22,573::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:33:22,574::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:33:22,574::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e071e4cf-68b5-49f3-83f8-b45307a32139`::Granted request >Thread-46::DEBUG::2014-08-22 09:33:22,574::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`50a09a17-0453-488a-8bb6-cfd22265a378`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:33:22,574::task::993::Storage.TaskManager.Task::(_decref) Task=`50a09a17-0453-488a-8bb6-cfd22265a378`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:33:22,576::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:22,576::task::1191::Storage.TaskManager.Task::(prepare) Task=`50a09a17-0453-488a-8bb6-cfd22265a378`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:22,576::task::595::Storage.TaskManager.Task::(_updateState) Task=`50a09a17-0453-488a-8bb6-cfd22265a378`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:33:22,576::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:33:22,576::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:33:22,576::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:33:22,576::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:33:22,577::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:33:22,577::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:33:22,577::task::993::Storage.TaskManager.Task::(_decref) Task=`50a09a17-0453-488a-8bb6-cfd22265a378`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:23,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:23,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118051 s, 86.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:23,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`db45de7b-d17a-4586-b59f-8840edfc7c83`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:23,441::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:33:23,441::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000146485', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216411', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:23,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`db45de7b-d17a-4586-b59f-8840edfc7c83`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000146485', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216411', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:23,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`db45de7b-d17a-4586-b59f-8840edfc7c83`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:23,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:23,441::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:23,441::task::993::Storage.TaskManager.Task::(_decref) Task=`db45de7b-d17a-4586-b59f-8840edfc7c83`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:25,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:25,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116745 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:27,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:27,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117512 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:27,333::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,334::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:27,334::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,335::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,335::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:27,336::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,339::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:27,339::__init__::225::IOProcess::(_processLogs) (771) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:33:27,340::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:27,340::__init__::225::IOProcess::(_processLogs) (771) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:27,347::__init__::225::IOProcess::(_processLogs) (771) Building response >Thread-21::DEBUG::2014-08-22 09:33:27,348::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00024201 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:27,348::__init__::225::IOProcess::(_processLogs) (771) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,349::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,350::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:27,351::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,352::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:27,353::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,354::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,354::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:27,355::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,356::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:27,356::__init__::225::IOProcess::(_processLogs) (772) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:27,357::__init__::225::IOProcess::(_processLogs) (772) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:27,360::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(772) Building response >Thread-17::DEBUG::2014-08-22 09:33:27,361::__init__::225::IOProcess::(_processLogs) (772) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,362::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,362::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:27,821::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,821::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:27,821::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,822::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,822::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:33:27,822::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:27,823::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:33:27,830::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000237825 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:27,830::__init__::225::IOProcess::(_processLogs) (773) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:27,831::__init__::225::IOProcess::(_processLogs) (773) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:27,831::__init__::225::IOProcess::(_processLogs) (773) Building response >Thread-17::DEBUG::2014-08-22 09:33:27,831::__init__::225::IOProcess::(_processLogs) (773) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,831::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,831::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) (774) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:27,832::__init__::225::IOProcess::(_processLogs) (774) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:27,833::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(774) Building response >Thread-17::DEBUG::2014-08-22 09:33:27,834::__init__::225::IOProcess::(_processLogs) (774) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,834::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,834::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:27,834::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,834::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:33:27,834::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) (775) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) (775) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) (775) Building response >Thread-17::DEBUG::2014-08-22 09:33:27,835::__init__::225::IOProcess::(_processLogs) (775) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,836::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,836::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:27,836::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,836::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:33:27,836::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,836::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,837::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:27,837::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) (776) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) (776) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) (776) Building response >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) (776) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:33:27,838::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:27,839::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:27,839::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:27,839::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) (777) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) (777) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) (777) Building response >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) (777) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:27,840::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:33:29,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:29,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122703 s, 83.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:31,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:31,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117536 s, 87.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:32,626::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:33:32,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`1efa41f3-9007-40c2-a05b-d282a0280556`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:32,629::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:33:32,630::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:32,631::task::1191::Storage.TaskManager.Task::(prepare) Task=`1efa41f3-9007-40c2-a05b-d282a0280556`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:32,632::task::595::Storage.TaskManager.Task::(_updateState) Task=`1efa41f3-9007-40c2-a05b-d282a0280556`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:32,632::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:32,633::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:32,633::task::993::Storage.TaskManager.Task::(_decref) Task=`1efa41f3-9007-40c2-a05b-d282a0280556`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:33:32,719::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:33:32,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`889a0cac-0875-48c9-b3e3-c893eebc42bb`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:33:32,721::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:33:32,723::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ecd22647-b46b-4436-ac58-6a8b0b468d06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:33:32,723::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:33:32,724::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:33:32,725::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ecd22647-b46b-4436-ac58-6a8b0b468d06`::Granted request >Thread-46::DEBUG::2014-08-22 09:33:32,727::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`889a0cac-0875-48c9-b3e3-c893eebc42bb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:33:32,727::task::993::Storage.TaskManager.Task::(_decref) Task=`889a0cac-0875-48c9-b3e3-c893eebc42bb`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:33:32,742::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:32,742::task::1191::Storage.TaskManager.Task::(prepare) Task=`889a0cac-0875-48c9-b3e3-c893eebc42bb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:32,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`889a0cac-0875-48c9-b3e3-c893eebc42bb`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:33:32,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:33:32,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:33:32,745::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:33:32,746::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:33:32,747::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:33:32,747::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:33:32,748::task::993::Storage.TaskManager.Task::(_decref) Task=`889a0cac-0875-48c9-b3e3-c893eebc42bb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:33,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:33,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116406 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:35,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:35,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116765 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:37,361::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,361::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) (778) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:37,362::__init__::225::IOProcess::(_processLogs) (778) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:37,363::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(778) Building response >Thread-21::DEBUG::2014-08-22 09:33:37,363::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:37,363::__init__::225::IOProcess::(_processLogs) (778) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,369::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,370::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:33:37,370::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000222269 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:37,371::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,371::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:37,371::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,372::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,372::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,372::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,372::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,372::__init__::225::IOProcess::(_processLogs) (779) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:37,372::__init__::225::IOProcess::(_processLogs) (779) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:37,373::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(779) Building response >Thread-17::DEBUG::2014-08-22 09:33:37,373::__init__::225::IOProcess::(_processLogs) (779) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,374::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,374::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:33:37,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:37,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117592 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:37,840::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,840::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:37,840::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,840::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,840::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) (780) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) (780) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) (780) Building response >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) (780) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,841::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,842::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:33:37,842::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:33:37,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000225355 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:37,849::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,849::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:37,849::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,849::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,850::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,850::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,850::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,850::__init__::225::IOProcess::(_processLogs) (781) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:37,850::__init__::225::IOProcess::(_processLogs) (781) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) (781) Building response >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) (781) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:33:37,851::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) (782) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) (782) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) (782) Building response >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) (782) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,852::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:37,853::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,853::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:33:37,853::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,853::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) (783) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) (783) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) (783) Building response >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) (783) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:37,854::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:37,855::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:33:37,855::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:37,855::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:37,855::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) (784) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) (784) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) (784) Building response >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) (784) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:37,856::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:33:38,630::task::595::Storage.TaskManager.Task::(_updateState) Task=`71421641-3749-41e4-9098-4694c98571d5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:38,630::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:33:38,630::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000222269', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225355', 'lastCheck': '0.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:38,631::task::1191::Storage.TaskManager.Task::(prepare) Task=`71421641-3749-41e4-9098-4694c98571d5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000222269', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225355', 'lastCheck': '0.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:38,631::task::595::Storage.TaskManager.Task::(_updateState) Task=`71421641-3749-41e4-9098-4694c98571d5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:38,631::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:38,631::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:38,631::task::993::Storage.TaskManager.Task::(_decref) Task=`71421641-3749-41e4-9098-4694c98571d5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:39,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:39,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119774 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:41,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:41,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118741 s, 86.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:42,796::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:33:42,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbef0893-9d75-42f1-8eb1-bdf9920bdb2d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:42,796::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:33:42,797::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:42,797::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbef0893-9d75-42f1-8eb1-bdf9920bdb2d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:42,797::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbef0893-9d75-42f1-8eb1-bdf9920bdb2d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:42,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:42,797::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:42,797::task::993::Storage.TaskManager.Task::(_decref) Task=`cbef0893-9d75-42f1-8eb1-bdf9920bdb2d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:33:42,835::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:33:42,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a6d96e3-c352-4a1c-a5f2-9ef3195d8d15`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:33:42,835::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:33:42,836::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5554c513-153e-4b9f-a8b4-4574a5c727c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:33:42,836::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:33:42,836::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:33:42,836::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5554c513-153e-4b9f-a8b4-4574a5c727c9`::Granted request >Thread-46::DEBUG::2014-08-22 09:33:42,836::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8a6d96e3-c352-4a1c-a5f2-9ef3195d8d15`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:33:42,836::task::993::Storage.TaskManager.Task::(_decref) Task=`8a6d96e3-c352-4a1c-a5f2-9ef3195d8d15`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:33:42,838::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:42,838::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a6d96e3-c352-4a1c-a5f2-9ef3195d8d15`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:42,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a6d96e3-c352-4a1c-a5f2-9ef3195d8d15`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:33:42,838::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:33:42,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:33:42,839::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:33:42,839::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:33:42,839::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:33:42,839::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:33:42,839::task::993::Storage.TaskManager.Task::(_decref) Task=`8a6d96e3-c352-4a1c-a5f2-9ef3195d8d15`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:43,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:43,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117044 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:45,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:45,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116238 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:47,374::__init__::225::IOProcess::(_processLogs) (785) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:47,375::__init__::225::IOProcess::(_processLogs) (785) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:47,375::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(785) Building response >Thread-21::DEBUG::2014-08-22 09:33:47,376::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:47,376::__init__::225::IOProcess::(_processLogs) (785) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,382::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:33:47,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000237801 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:47,384::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:47,384::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,384::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) (786) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:47,385::__init__::225::IOProcess::(_processLogs) (786) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:47,386::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(786) Building response >Thread-17::DEBUG::2014-08-22 09:33:47,387::__init__::225::IOProcess::(_processLogs) (786) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,387::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:47,387::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:33:47,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:47,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118473 s, 86.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:47,872::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,872::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:47,873::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,874::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,874::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,875::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,877::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Extracting request information... >Thread-22::DEBUG::2014-08-22 09:33:47,878::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:47,878::__init__::225::IOProcess::(_processLogs) (787) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:47,885::__init__::225::IOProcess::(_processLogs) (787) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 09:33:47,886::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000156334 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:47,886::__init__::225::IOProcess::(_processLogs) (787) Building response >Thread-17::DEBUG::2014-08-22 09:33:47,888::__init__::225::IOProcess::(_processLogs) (787) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,888::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:47,889::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:47,890::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,890::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:47,890::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,891::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,891::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,893::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,894::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:47,895::__init__::225::IOProcess::(_processLogs) (788) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:47,895::__init__::225::IOProcess::(_processLogs) (788) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:47,895::__init__::225::IOProcess::(_processLogs) (788) Building response >Thread-17::DEBUG::2014-08-22 09:33:47,896::__init__::225::IOProcess::(_processLogs) (788) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,896::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:47,896::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:47,897::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,898::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:33:47,898::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,898::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,899::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,900::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,900::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:47,901::__init__::225::IOProcess::(_processLogs) (789) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:47,901::__init__::225::IOProcess::(_processLogs) (789) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:47,902::__init__::225::IOProcess::(_processLogs) (789) Building response >Thread-17::DEBUG::2014-08-22 09:33:47,902::__init__::225::IOProcess::(_processLogs) (789) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,902::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:47,903::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:47,904::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,904::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:33:47,904::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,905::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,905::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,907::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:47,907::__init__::225::IOProcess::(_processLogs) (790) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:47,908::__init__::225::IOProcess::(_processLogs) (790) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:47,908::__init__::225::IOProcess::(_processLogs) (790) Building response >Thread-17::DEBUG::2014-08-22 09:33:47,908::__init__::225::IOProcess::(_processLogs) (790) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,909::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:47,909::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:47,910::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:33:47,910::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:33:47,911::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:47,911::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:47,911::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:47,912::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:47,913::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:47,914::__init__::225::IOProcess::(_processLogs) (791) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:47,914::__init__::225::IOProcess::(_processLogs) (791) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:47,914::__init__::225::IOProcess::(_processLogs) (791) Building response >Thread-17::DEBUG::2014-08-22 09:33:47,915::__init__::225::IOProcess::(_processLogs) (791) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:47,915::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:47,915::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:33:49,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:49,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011603 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:51,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:52,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140623 s, 72.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:33:52,884::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:33:52,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`934198e4-197b-4bec-ad55-3c7d046837a9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:52,884::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:33:52,884::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:52,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`934198e4-197b-4bec-ad55-3c7d046837a9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:33:52,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`934198e4-197b-4bec-ad55-3c7d046837a9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:52,885::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:52,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:52,885::task::993::Storage.TaskManager.Task::(_decref) Task=`934198e4-197b-4bec-ad55-3c7d046837a9`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:33:52,925::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:33:52,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`647d0733-8170-4717-9c1a-acfbb74a91ef`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:33:52,925::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:33:52,925::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a1b48bbf-9417-4d25-9e8d-6bb1f78c0754`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:33:52,926::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:33:52,926::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:33:52,926::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a1b48bbf-9417-4d25-9e8d-6bb1f78c0754`::Granted request >Thread-46::DEBUG::2014-08-22 09:33:52,926::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`647d0733-8170-4717-9c1a-acfbb74a91ef`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:33:52,926::task::993::Storage.TaskManager.Task::(_decref) Task=`647d0733-8170-4717-9c1a-acfbb74a91ef`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:33:52,928::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:52,928::task::1191::Storage.TaskManager.Task::(prepare) Task=`647d0733-8170-4717-9c1a-acfbb74a91ef`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:33:52,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`647d0733-8170-4717-9c1a-acfbb74a91ef`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:33:52,928::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:33:52,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:33:52,929::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:33:52,929::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:33:52,929::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:33:52,929::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:33:52,929::task::993::Storage.TaskManager.Task::(_decref) Task=`647d0733-8170-4717-9c1a-acfbb74a91ef`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:33:53,822::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5a5e94d-c287-4b2b-8b73-c4d81f345bfb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:33:53,823::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:33:53,823::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000237801', 'lastCheck': '6.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000156334', 'lastCheck': '5.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:53,823::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5a5e94d-c287-4b2b-8b73-c4d81f345bfb`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000237801', 'lastCheck': '6.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000156334', 'lastCheck': '5.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:33:53,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5a5e94d-c287-4b2b-8b73-c4d81f345bfb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:33:53,823::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:33:53,823::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:33:53,823::task::993::Storage.TaskManager.Task::(_decref) Task=`d5a5e94d-c287-4b2b-8b73-c4d81f345bfb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:33:54,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:54,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117399 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:33:56,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:56,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115302 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:57,387::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:57,387::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:57,387::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,387::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,387::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,387::__init__::225::IOProcess::(_processLogs) (792) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:57,388::__init__::225::IOProcess::(_processLogs) (792) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:57,388::__init__::225::IOProcess::(_processLogs) (792) Queuing response >Thread-21::DEBUG::2014-08-22 09:33:57,389::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:57,389::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,395::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:33:57,397::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.0013049 s, 304 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:57,397::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:33:57,397::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:57,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,398::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,398::__init__::225::IOProcess::(_processLogs) (793) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:57,398::__init__::225::IOProcess::(_processLogs) (793) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:57,398::__init__::225::IOProcess::(_processLogs) (793) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:57,399::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,399::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:57,914::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:57,914::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:57,914::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,914::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,914::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,914::__init__::225::IOProcess::(_processLogs) (794) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:57,915::__init__::225::IOProcess::(_processLogs) (794) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:57,915::__init__::225::IOProcess::(_processLogs) (794) Queuing response >Thread-22::DEBUG::2014-08-22 09:33:57,916::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:33:57,916::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,922::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:33:57,922::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000204116 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) (795) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:33:57,923::__init__::225::IOProcess::(_processLogs) (795) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:33:57,924::__init__::225::IOProcess::(_processLogs) (795) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:57,925::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,925::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:33:57,925::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:33:57,925::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:57,925::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) (796) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) (796) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) (796) Building response >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) (796) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,926::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) (797) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:57,927::__init__::225::IOProcess::(_processLogs) (797) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) (797) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:33:57,928::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:33:57,929::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:33:57,929::__init__::225::IOProcess::(_processLogs) (798) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:33:57,929::__init__::225::IOProcess::(_processLogs) (798) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:33:57,929::__init__::225::IOProcess::(_processLogs) (798) Queuing response >Thread-17::DEBUG::2014-08-22 09:33:57,929::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:33:57,930::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:33:58,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:33:58,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116127 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:00,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:00,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115674 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:02,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:02,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021852 s, 46.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:03,009::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:34:03,009::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad4db267-f364-4434-ba75-b37e04607c6f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:03,009::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:34:03,009::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:03,010::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad4db267-f364-4434-ba75-b37e04607c6f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:03,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad4db267-f364-4434-ba75-b37e04607c6f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:03,010::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:03,010::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:03,010::task::993::Storage.TaskManager.Task::(_decref) Task=`ad4db267-f364-4434-ba75-b37e04607c6f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:34:03,088::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:34:03,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d075e62-d495-472f-9524-160baa052975`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:34:03,088::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:34:03,088::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`deb45b71-5ad6-45e3-a11e-3d81a372309e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:34:03,088::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:34:03,089::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:34:03,089::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`deb45b71-5ad6-45e3-a11e-3d81a372309e`::Granted request >Thread-46::DEBUG::2014-08-22 09:34:03,089::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3d075e62-d495-472f-9524-160baa052975`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:34:03,089::task::993::Storage.TaskManager.Task::(_decref) Task=`3d075e62-d495-472f-9524-160baa052975`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:34:03,091::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:03,091::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d075e62-d495-472f-9524-160baa052975`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:03,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d075e62-d495-472f-9524-160baa052975`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:34:03,091::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:34:03,091::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:34:03,091::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:34:03,091::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:34:03,092::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:34:03,092::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:34:03,092::task::993::Storage.TaskManager.Task::(_decref) Task=`3d075e62-d495-472f-9524-160baa052975`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:04,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:04,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115464 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:06,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:06,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115887 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) (799) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:07,400::__init__::225::IOProcess::(_processLogs) (799) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:07,401::__init__::225::IOProcess::(_processLogs) (799) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,402::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:34:07,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:07,402::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:34:07,408::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000170586 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) (800) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:07,409::__init__::225::IOProcess::(_processLogs) (800) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:07,410::__init__::225::IOProcess::(_processLogs) DEBUG|(800) Building response >Thread-17::DEBUG::2014-08-22 09:34:07,411::__init__::225::IOProcess::(_processLogs) (800) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,411::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:07,411::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:07,930::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:07,930::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:07,930::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:07,931::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,932::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:34:07,932::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:07,932::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,938::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:34:07,939::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216497 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:07,940::__init__::225::IOProcess::(_processLogs) (801) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:07,940::__init__::225::IOProcess::(_processLogs) (801) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:07,940::__init__::225::IOProcess::(_processLogs) (801) Building response >Thread-17::DEBUG::2014-08-22 09:34:07,940::__init__::225::IOProcess::(_processLogs) (801) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,940::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:07,940::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) (802) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) (802) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:07,941::__init__::225::IOProcess::(_processLogs) (802) Building response >Thread-17::DEBUG::2014-08-22 09:34:07,942::__init__::225::IOProcess::(_processLogs) (802) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,943::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:07,943::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:07,943::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:07,943::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:34:07,943::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:07,943::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) (803) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) (803) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) (803) Building response >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) (803) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:07,944::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:07,945::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:07,945::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:34:07,945::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:07,945::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,945::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:07,945::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) (804) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) (804) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) (804) Building response >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) (804) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:34:07,946::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:07,947::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:07,947::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:07,947::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:07,947::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:07,948::__init__::225::IOProcess::(_processLogs) (805) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:07,948::__init__::225::IOProcess::(_processLogs) (805) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:07,948::__init__::225::IOProcess::(_processLogs) (805) Building response >Thread-17::DEBUG::2014-08-22 09:34:07,948::__init__::225::IOProcess::(_processLogs) (805) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:07,948::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:07,948::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:34:08,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:08,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122341 s, 83.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:09,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1c7daee-b582-4361-ae53-e328c66baf36`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:09,016::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:34:09,016::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000170586', 'lastCheck': '1.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216497', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:09,016::task::1191::Storage.TaskManager.Task::(prepare) Task=`f1c7daee-b582-4361-ae53-e328c66baf36`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000170586', 'lastCheck': '1.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216497', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:09,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1c7daee-b582-4361-ae53-e328c66baf36`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:09,016::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:09,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:09,017::task::993::Storage.TaskManager.Task::(_decref) Task=`f1c7daee-b582-4361-ae53-e328c66baf36`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:10,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:10,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139131 s, 73.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:12,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:12,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116178 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:13,141::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:34:13,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`042fd2a1-a6de-4528-8407-69c4e14100d1`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:13,142::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:34:13,142::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:13,142::task::1191::Storage.TaskManager.Task::(prepare) Task=`042fd2a1-a6de-4528-8407-69c4e14100d1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:13,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`042fd2a1-a6de-4528-8407-69c4e14100d1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:13,142::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:13,142::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:13,142::task::993::Storage.TaskManager.Task::(_decref) Task=`042fd2a1-a6de-4528-8407-69c4e14100d1`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:34:13,184::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:34:13,184::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e847cd5-62c3-4ed1-8627-315ea1d276d4`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:34:13,184::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:34:13,184::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7353968f-04f5-4196-b5df-5a129e5679d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:34:13,184::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:34:13,184::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:34:13,185::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7353968f-04f5-4196-b5df-5a129e5679d1`::Granted request >Thread-46::DEBUG::2014-08-22 09:34:13,185::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5e847cd5-62c3-4ed1-8627-315ea1d276d4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:34:13,185::task::993::Storage.TaskManager.Task::(_decref) Task=`5e847cd5-62c3-4ed1-8627-315ea1d276d4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:34:13,187::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:13,187::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e847cd5-62c3-4ed1-8627-315ea1d276d4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:13,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e847cd5-62c3-4ed1-8627-315ea1d276d4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:34:13,187::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:34:13,187::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:34:13,187::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:34:13,187::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:34:13,188::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:34:13,188::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:34:13,188::task::993::Storage.TaskManager.Task::(_decref) Task=`5e847cd5-62c3-4ed1-8627-315ea1d276d4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:14,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:14,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011571 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:16,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:16,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116301 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:17,411::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) (806) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:17,412::__init__::225::IOProcess::(_processLogs) (806) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:17,413::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|(806) Building response >Thread-21::DEBUG::2014-08-22 09:34:17,413::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:17,414::__init__::225::IOProcess::(_processLogs) (806) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,420::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:34:17,420::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199905 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:17,420::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) (807) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:17,422::__init__::225::IOProcess::(_processLogs) (807) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:17,423::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|(807) Building response >Thread-17::DEBUG::2014-08-22 09:34:17,424::__init__::225::IOProcess::(_processLogs) (807) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,424::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:17,424::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,948::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,949::__init__::225::IOProcess::(_processLogs) (808) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:17,949::__init__::225::IOProcess::(_processLogs) (808) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:17,949::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|(808) Building response >Thread-22::DEBUG::2014-08-22 09:34:17,950::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:17,950::__init__::225::IOProcess::(_processLogs) (808) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,956::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:34:17,956::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000146093 s, 3.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:17,957::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) (802) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) (809) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:17,958::__init__::225::IOProcess::(_processLogs) (809) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:17,959::__init__::225::IOProcess::(_processLogs) (809) Building response >Thread-17::DEBUG::2014-08-22 09:34:17,959::__init__::225::IOProcess::(_processLogs) (809) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,959::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:17,959::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:17,960::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,960::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:34:17,960::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) (810) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) (810) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) (810) Building response >Thread-17::DEBUG::2014-08-22 09:34:17,961::__init__::225::IOProcess::(_processLogs) (810) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,962::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:17,962::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:17,962::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,962::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:34:17,962::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) (811) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) (811) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) (811) Building response >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) (811) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,964::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:17,965::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:17,965::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:17,965::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:34:17,965::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:17,965::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:17,966::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:17,966::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:17,966::__init__::225::IOProcess::(_processLogs) (812) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:17,966::__init__::225::IOProcess::(_processLogs) (812) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:17,966::__init__::225::IOProcess::(_processLogs) (812) Building response >Thread-17::DEBUG::2014-08-22 09:34:17,966::__init__::225::IOProcess::(_processLogs) (812) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:17,967::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:17,967::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:34:18,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:18,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012013 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:20,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:20,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116346 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:22,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:22,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116776 s, 87.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:23,236::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:34:23,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`d51e7c53-3302-4e98-95ec-4599b64f30d1`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:23,236::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:34:23,236::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:23,237::task::1191::Storage.TaskManager.Task::(prepare) Task=`d51e7c53-3302-4e98-95ec-4599b64f30d1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:23,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`d51e7c53-3302-4e98-95ec-4599b64f30d1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:23,237::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:23,237::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:23,237::task::993::Storage.TaskManager.Task::(_decref) Task=`d51e7c53-3302-4e98-95ec-4599b64f30d1`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:34:23,291::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:34:23,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e08c274-0551-4447-b868-6c9b952778cd`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:34:23,291::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:34:23,291::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1153fd16-6696-45fa-b83b-3086918beb9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:34:23,291::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:34:23,291::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:34:23,292::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1153fd16-6696-45fa-b83b-3086918beb9c`::Granted request >Thread-46::DEBUG::2014-08-22 09:34:23,292::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1e08c274-0551-4447-b868-6c9b952778cd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:34:23,292::task::993::Storage.TaskManager.Task::(_decref) Task=`1e08c274-0551-4447-b868-6c9b952778cd`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:34:23,294::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:23,294::task::1191::Storage.TaskManager.Task::(prepare) Task=`1e08c274-0551-4447-b868-6c9b952778cd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:23,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e08c274-0551-4447-b868-6c9b952778cd`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:34:23,294::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:34:23,294::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:34:23,294::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:34:23,294::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:34:23,294::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:34:23,294::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:34:23,295::task::993::Storage.TaskManager.Task::(_decref) Task=`1e08c274-0551-4447-b868-6c9b952778cd`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:34:24,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`25c379a4-fe92-465d-845d-387957dd148f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:24,209::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:34:24,209::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199905', 'lastCheck': '6.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000146093', 'lastCheck': '6.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:24,209::task::1191::Storage.TaskManager.Task::(prepare) Task=`25c379a4-fe92-465d-845d-387957dd148f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199905', 'lastCheck': '6.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000146093', 'lastCheck': '6.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:24,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`25c379a4-fe92-465d-845d-387957dd148f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:24,209::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:24,209::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:24,209::task::993::Storage.TaskManager.Task::(_decref) Task=`25c379a4-fe92-465d-845d-387957dd148f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:24,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:24,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118729 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:26,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:27,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115575 s, 88.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:27,424::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) (813) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) (813) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) (813) Building response >Thread-17::DEBUG::2014-08-22 09:34:27,425::__init__::225::IOProcess::(_processLogs) (813) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,426::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:34:27,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:27,427::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:34:27,433::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000321251 s, 1.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Sending response sized 328DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:27,434::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,435::__init__::225::IOProcess::(_processLogs) (814) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:27,435::__init__::225::IOProcess::(_processLogs) (814) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:27,435::__init__::225::IOProcess::(_processLogs) (814) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,436::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:27,436::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:27,966::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:27,967::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:27,967::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:27,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:34:27,968::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:27,969::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,975::__init__::225::IOProcess::(_processLogs) (815) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:34:27,975::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000226109 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:27,976::__init__::225::IOProcess::(_processLogs) (815) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:27,976::__init__::225::IOProcess::(_processLogs) (815) Building response >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) (815) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,977::__init__::225::IOProcess::(_processLogs) (816) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:27,978::__init__::225::IOProcess::(_processLogs) (816) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:27,978::__init__::225::IOProcess::(_processLogs) (816) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,979::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:27,979::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:27,979::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:34:27,979::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) (817) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) (817) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) (817) Building response >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) (817) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:27,980::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:27,981::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:34:27,981::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:27,981::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:27,981::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,981::__init__::225::IOProcess::(_processLogs) (818) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:27,981::__init__::225::IOProcess::(_processLogs) (818) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:27,982::__init__::225::IOProcess::(_processLogs) (818) Building response >Thread-17::DEBUG::2014-08-22 09:34:27,982::__init__::225::IOProcess::(_processLogs) (818) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,982::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:27,982::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:27,982::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:34:27,982::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) (819) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) (819) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) (819) Building response >Thread-17::DEBUG::2014-08-22 09:34:27,983::__init__::225::IOProcess::(_processLogs) (819) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:27,984::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:27,984::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:34:29,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:29,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012163 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:31,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:31,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116993 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:33,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:33,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130151 s, 78.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:33,405::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:34:33,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`762e1e9a-31de-41bc-af38-f50981c15479`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:33,406::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:34:33,406::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:33,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`762e1e9a-31de-41bc-af38-f50981c15479`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:33,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`762e1e9a-31de-41bc-af38-f50981c15479`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:33,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:33,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:33,406::task::993::Storage.TaskManager.Task::(_decref) Task=`762e1e9a-31de-41bc-af38-f50981c15479`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:34:33,440::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:34:33,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`5db02809-5aa8-45eb-b841-3af5affa5472`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:34:33,440::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:34:33,440::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ea81d2f6-84ee-4d4b-892b-93b6733660b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:34:33,441::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:34:33,441::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:34:33,441::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ea81d2f6-84ee-4d4b-892b-93b6733660b4`::Granted request >Thread-46::DEBUG::2014-08-22 09:34:33,441::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5db02809-5aa8-45eb-b841-3af5affa5472`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:34:33,441::task::993::Storage.TaskManager.Task::(_decref) Task=`5db02809-5aa8-45eb-b841-3af5affa5472`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:34:33,443::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:33,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`5db02809-5aa8-45eb-b841-3af5affa5472`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:33,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`5db02809-5aa8-45eb-b841-3af5affa5472`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:34:33,443::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:34:33,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:34:33,444::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:34:33,444::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:34:33,444::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:34:33,444::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:34:33,444::task::993::Storage.TaskManager.Task::(_decref) Task=`5db02809-5aa8-45eb-b841-3af5affa5472`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:35,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:35,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118825 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:37,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:37,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194116 s, 52.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) (820) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:37,437::__init__::225::IOProcess::(_processLogs) (820) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:37,438::__init__::225::IOProcess::(_processLogs) (820) Queuing response >Thread-21::DEBUG::2014-08-22 09:34:37,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:37,439::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:37,445::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:34:37,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000233536 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:37,446::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) (821) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:37,447::__init__::225::IOProcess::(_processLogs) (821) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:37,448::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(821) Building response >Thread-17::DEBUG::2014-08-22 09:34:37,449::__init__::225::IOProcess::(_processLogs) (821) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:37,449::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:37,449::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) (822) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:37,984::__init__::225::IOProcess::(_processLogs) (822) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:37,985::__init__::225::IOProcess::(_processLogs) (822) Building response >Thread-17::DEBUG::2014-08-22 09:34:37,985::__init__::225::IOProcess::(_processLogs) (822) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:37,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:37,985::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:34:37,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:34:37,991::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000225896 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:37,992::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:37,992::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:37,992::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,993::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) (823) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) (823) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) (823) Building response >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) (823) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:37,994::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:37,995::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:37,995::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:34:37,995::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,995::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:37,995::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) (824) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) (824) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) (824) Building response >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) (824) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:37,996::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:34:37,997::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,997::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:37,997::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,997::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,997::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,997::__init__::225::IOProcess::(_processLogs) (825) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) (825) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) (825) Building response >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) (825) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:34:37,998::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:37,999::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:37,999::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:37,999::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:37,999::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:37,999::__init__::225::IOProcess::(_processLogs) (826) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:38,000::__init__::225::IOProcess::(_processLogs) (826) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:38,000::__init__::225::IOProcess::(_processLogs) (826) Building response >Thread-17::DEBUG::2014-08-22 09:34:38,000::__init__::225::IOProcess::(_processLogs) (826) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:38,000::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:38,000::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:34:39,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:39,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123804 s, 82.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:39,380::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d87ee0a-e4fc-4661-924a-b6e8637b1fac`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:39,381::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:34:39,381::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233536', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225896', 'lastCheck': '1.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:39,381::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d87ee0a-e4fc-4661-924a-b6e8637b1fac`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233536', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225896', 'lastCheck': '1.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:39,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d87ee0a-e4fc-4661-924a-b6e8637b1fac`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:39,381::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:39,381::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:39,381::task::993::Storage.TaskManager.Task::(_decref) Task=`7d87ee0a-e4fc-4661-924a-b6e8637b1fac`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:41,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:41,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118229 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:43,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:43,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195814 s, 52.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:43,510::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:34:43,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`1870efb7-1268-414b-a7a1-6e3bc66ce993`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:43,510::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:34:43,511::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:43,511::task::1191::Storage.TaskManager.Task::(prepare) Task=`1870efb7-1268-414b-a7a1-6e3bc66ce993`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:43,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`1870efb7-1268-414b-a7a1-6e3bc66ce993`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:43,511::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:43,511::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:43,511::task::993::Storage.TaskManager.Task::(_decref) Task=`1870efb7-1268-414b-a7a1-6e3bc66ce993`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:34:43,595::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:34:43,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f752f63-5e95-4d5b-a6e4-7e9de4f97508`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:34:43,596::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:34:43,596::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fbb1ee0c-18e3-4436-b022-7986211fbb2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:34:43,596::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:34:43,596::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:34:43,596::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fbb1ee0c-18e3-4436-b022-7986211fbb2e`::Granted request >Thread-46::DEBUG::2014-08-22 09:34:43,597::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0f752f63-5e95-4d5b-a6e4-7e9de4f97508`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:34:43,597::task::993::Storage.TaskManager.Task::(_decref) Task=`0f752f63-5e95-4d5b-a6e4-7e9de4f97508`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:34:43,599::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:43,599::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f752f63-5e95-4d5b-a6e4-7e9de4f97508`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:43,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f752f63-5e95-4d5b-a6e4-7e9de4f97508`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:34:43,599::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:34:43,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:34:43,599::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:34:43,599::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:34:43,599::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:34:43,600::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:34:43,600::task::993::Storage.TaskManager.Task::(_decref) Task=`0f752f63-5e95-4d5b-a6e4-7e9de4f97508`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:45,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:45,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117298 s, 87.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:47,449::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:47,449::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:47,449::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:47,449::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:47,449::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:47,450::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:47,450::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:47,450::__init__::225::IOProcess::(_processLogs) (827) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:47,450::__init__::225::IOProcess::(_processLogs) (827) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:47,450::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(827) Building response >Thread-21::DEBUG::2014-08-22 09:34:47,451::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:47,451::__init__::225::IOProcess::(_processLogs) (827) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:47,457::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:34:47,459::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196423 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:47,459::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:47,460::__init__::225::IOProcess::(_processLogs) (828) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:47,461::__init__::225::IOProcess::(_processLogs) (828) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:47,461::__init__::225::IOProcess::(_processLogs) DEBUG|(828) Building response >Thread-17::DEBUG::2014-08-22 09:34:47,462::__init__::225::IOProcess::(_processLogs) (828) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:47,462::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:47,462::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:34:47,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:47,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117996 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:48,000::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:48,000::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:48,000::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:48,000::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:48,000::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:48,000::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:48,001::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:48,001::__init__::225::IOProcess::(_processLogs) (829) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:48,001::__init__::225::IOProcess::(_processLogs) (829) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:48,001::__init__::225::IOProcess::(_processLogs) (829) Building response >Thread-17::DEBUG::2014-08-22 09:34:48,001::__init__::225::IOProcess::(_processLogs) (829) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:48,001::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:48,002::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:34:48,002::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:34:48,008::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000270038 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:48,009::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:48,009::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:48,009::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:48,010::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:48,010::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:48,010::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:48,010::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) (830) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) (830) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) (830) Building response >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) (830) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:34:48,011::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:48,012::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:48,012::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:48,012::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:48,012::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:48,012::__init__::225::IOProcess::(_processLogs) (831) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:48,012::__init__::225::IOProcess::(_processLogs) (831) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) (831) Building response >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) (831) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:34:48,013::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) (832) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) (832) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) (832) Building response >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) (832) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:48,014::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:48,015::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:48,015::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:34:48,015::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:48,015::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) (833) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) (833) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) (833) Building response >Thread-17::DEBUG::2014-08-22 09:34:48,016::__init__::225::IOProcess::(_processLogs) (833) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:48,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:48,017::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:34:49,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:49,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117387 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:34:51,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:51,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012937 s, 79.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:34:53,648::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:34:53,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2f812ba-4ad7-46db-b519-45ee8a4c7447`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:53,649::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:34:53,649::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:53,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2f812ba-4ad7-46db-b519-45ee8a4c7447`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:34:53,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2f812ba-4ad7-46db-b519-45ee8a4c7447`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:53,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:53,650::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:53,650::task::993::Storage.TaskManager.Task::(_decref) Task=`f2f812ba-4ad7-46db-b519-45ee8a4c7447`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:53,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:53,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119839 s, 85.4 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:34:53,848::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:34:53,848::task::595::Storage.TaskManager.Task::(_updateState) Task=`9fe6d6b8-571d-4187-8c20-090d4bb41b99`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:34:53,848::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:34:53,849::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6ba0294b-8a09-479d-8fb6-4d064e342f34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:34:53,849::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:34:53,849::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:34:53,849::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6ba0294b-8a09-479d-8fb6-4d064e342f34`::Granted request >Thread-46::DEBUG::2014-08-22 09:34:53,849::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9fe6d6b8-571d-4187-8c20-090d4bb41b99`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:34:53,849::task::993::Storage.TaskManager.Task::(_decref) Task=`9fe6d6b8-571d-4187-8c20-090d4bb41b99`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:34:53,851::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:53,851::task::1191::Storage.TaskManager.Task::(prepare) Task=`9fe6d6b8-571d-4187-8c20-090d4bb41b99`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:34:53,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`9fe6d6b8-571d-4187-8c20-090d4bb41b99`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:34:53,852::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:34:53,852::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:34:53,852::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:34:53,852::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:34:53,852::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:34:53,852::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:34:53,852::task::993::Storage.TaskManager.Task::(_decref) Task=`9fe6d6b8-571d-4187-8c20-090d4bb41b99`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:34:54,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`feddfc05-be2a-4595-b6de-a1e5954e6f8c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:34:54,669::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:34:54,670::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196423', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000270038', 'lastCheck': '6.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:54,670::task::1191::Storage.TaskManager.Task::(prepare) Task=`feddfc05-be2a-4595-b6de-a1e5954e6f8c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196423', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000270038', 'lastCheck': '6.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:34:54,670::task::595::Storage.TaskManager.Task::(_updateState) Task=`feddfc05-be2a-4595-b6de-a1e5954e6f8c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:34:54,670::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:34:54,670::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:34:54,670::task::993::Storage.TaskManager.Task::(_decref) Task=`feddfc05-be2a-4595-b6de-a1e5954e6f8c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:34:55,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:55,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115569 s, 88.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:57,462::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:57,462::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:57,462::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:57,462::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:57,463::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:57,463::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:57,463::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:57,463::__init__::225::IOProcess::(_processLogs) (834) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:57,463::__init__::225::IOProcess::(_processLogs) (834) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:57,464::__init__::225::IOProcess::(_processLogs) DEBUG|(834) Building response >Thread-21::DEBUG::2014-08-22 09:34:57,464::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:57,464::__init__::225::IOProcess::(_processLogs) (834) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:57,470::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:57,471::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:34:57,471::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000176585 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:57,471::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:57,472::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:34:57,472::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:57,473::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:57,473::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) (835) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) (835) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) (835) Building response >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) (835) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:57,474::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:34:57,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:57,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118236 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:58,016::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) (836) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:58,017::__init__::225::IOProcess::(_processLogs) (836) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:58,018::__init__::225::IOProcess::(_processLogs) DEBUG|(836) Building response >Thread-22::DEBUG::2014-08-22 09:34:58,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:34:58,019::__init__::225::IOProcess::(_processLogs) (836) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:58,025::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:34:58,026::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000230235 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:34:58,026::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:58,027::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:58,027::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:34:58,028::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) (837) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) (837) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) (837) Building response >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) (837) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:58,029::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:34:58,030::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:58,030::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:34:58,030::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:58,030::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:58,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) (838) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) (838) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) (838) Building response >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) (838) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:34:58,031::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:58,032::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:58,032::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:58,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:58,032::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:58,032::__init__::225::IOProcess::(_processLogs) (839) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) (839) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) (839) Building response >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) (839) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:34:58,033::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:34:58,034::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:34:58,034::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:34:58,034::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:34:58,034::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:34:58,034::__init__::225::IOProcess::(_processLogs) (840) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:34:58,035::__init__::225::IOProcess::(_processLogs) (840) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:34:58,035::__init__::225::IOProcess::(_processLogs) (840) Building response >Thread-17::DEBUG::2014-08-22 09:34:58,035::__init__::225::IOProcess::(_processLogs) (840) Queuing response >Thread-17::DEBUG::2014-08-22 09:34:58,035::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:34:58,035::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:34:59,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:34:59,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116145 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:01,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:01,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011422 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:03,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:03,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118292 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:03,985::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:35:03,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b6a0380-5642-4c15-9de2-dd9ae42eb9ad`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:03,986::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:35:03,986::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:03,986::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b6a0380-5642-4c15-9de2-dd9ae42eb9ad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:03,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b6a0380-5642-4c15-9de2-dd9ae42eb9ad`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:03,986::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:03,986::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:03,986::task::993::Storage.TaskManager.Task::(_decref) Task=`2b6a0380-5642-4c15-9de2-dd9ae42eb9ad`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:35:04,035::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:35:04,035::task::595::Storage.TaskManager.Task::(_updateState) Task=`65f985c3-00fd-43d5-9933-316ccf72264c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:35:04,035::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:35:04,035::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3b6a374f-72e8-497e-83b0-6571542d1122`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:35:04,036::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:35:04,036::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:35:04,036::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3b6a374f-72e8-497e-83b0-6571542d1122`::Granted request >Thread-46::DEBUG::2014-08-22 09:35:04,036::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`65f985c3-00fd-43d5-9933-316ccf72264c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:35:04,036::task::993::Storage.TaskManager.Task::(_decref) Task=`65f985c3-00fd-43d5-9933-316ccf72264c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:35:04,038::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:04,038::task::1191::Storage.TaskManager.Task::(prepare) Task=`65f985c3-00fd-43d5-9933-316ccf72264c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:04,038::task::595::Storage.TaskManager.Task::(_updateState) Task=`65f985c3-00fd-43d5-9933-316ccf72264c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:35:04,038::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:35:04,039::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:35:04,039::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:35:04,039::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:35:04,039::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:35:04,039::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:35:04,039::task::993::Storage.TaskManager.Task::(_decref) Task=`65f985c3-00fd-43d5-9933-316ccf72264c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:05,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:06,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012072 s, 84.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:07,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:07,475::__init__::225::IOProcess::(_processLogs) (841) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:07,475::__init__::225::IOProcess::(_processLogs) (841) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:07,475::__init__::225::IOProcess::(_processLogs) DEBUG|(841) Building response >Thread-21::DEBUG::2014-08-22 09:35:07,476::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:07,476::__init__::225::IOProcess::(_processLogs) (841) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:07,482::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:35:07,483::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000234045 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:07,483::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:07,484::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) (842) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:07,485::__init__::225::IOProcess::(_processLogs) (842) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:07,486::__init__::225::IOProcess::(_processLogs) DEBUG|(842) Building response >Thread-17::DEBUG::2014-08-22 09:35:07,487::__init__::225::IOProcess::(_processLogs) (842) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:07,487::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:07,487::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:35:08,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:08,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119766 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:08,057::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:08,058::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:08,058::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:08,059::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:08,060::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:08,060::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:08,064::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:08,065::__init__::225::IOProcess::(_processLogs) (843) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:35:08,065::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:08,066::__init__::225::IOProcess::(_processLogs) (843) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:08,072::__init__::225::IOProcess::(_processLogs) (843) Building response >Thread-22::DEBUG::2014-08-22 09:35:08,073::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00015492 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:08,074::__init__::225::IOProcess::(_processLogs) (843) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:08,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:08,075::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:08,076::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:08,077::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:08,077::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:08,077::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:08,078::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:08,080::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:08,081::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:08,081::__init__::225::IOProcess::(_processLogs) (844) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:08,081::__init__::225::IOProcess::(_processLogs) (844) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:08,082::__init__::225::IOProcess::(_processLogs) (844) Building response >Thread-17::DEBUG::2014-08-22 09:35:08,082::__init__::225::IOProcess::(_processLogs) (844) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:08,083::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:08,083::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:08,084::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:08,084::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:35:08,084::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:08,085::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:08,085::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:08,086::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:08,087::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:08,087::__init__::225::IOProcess::(_processLogs) (845) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:08,088::__init__::225::IOProcess::(_processLogs) (845) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:08,088::__init__::225::IOProcess::(_processLogs) (845) Building response >Thread-17::DEBUG::2014-08-22 09:35:08,088::__init__::225::IOProcess::(_processLogs) (845) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:08,089::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:08,089::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:08,090::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:08,090::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:35:08,090::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:08,091::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:08,091::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:08,092::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:08,093::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:08,093::__init__::225::IOProcess::(_processLogs) (846) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:08,093::__init__::225::IOProcess::(_processLogs) (846) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:08,094::__init__::225::IOProcess::(_processLogs) (846) Building response >Thread-17::DEBUG::2014-08-22 09:35:08,094::__init__::225::IOProcess::(_processLogs) (846) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:08,094::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:08,095::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:08,096::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:35:08,096::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:08,096::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:08,097::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:08,098::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:08,099::__init__::225::IOProcess::(_processLogs) (847) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:08,099::__init__::225::IOProcess::(_processLogs) (847) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:08,099::__init__::225::IOProcess::(_processLogs) (847) Building response >Thread-17::DEBUG::2014-08-22 09:35:08,100::__init__::225::IOProcess::(_processLogs) (847) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:08,100::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:08,101::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-47::DEBUG::2014-08-22 09:35:09,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`95c595dd-0aae-410c-9178-109347c3080c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:09,884::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:35:09,885::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000234045', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00015492', 'lastCheck': '1.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:09,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`95c595dd-0aae-410c-9178-109347c3080c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000234045', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00015492', 'lastCheck': '1.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:09,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`95c595dd-0aae-410c-9178-109347c3080c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:09,885::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:09,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:09,885::task::993::Storage.TaskManager.Task::(_decref) Task=`95c595dd-0aae-410c-9178-109347c3080c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:10,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:10,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011923 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:12,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:12,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011506 s, 89.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:14,088::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:35:14,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2722f64-6567-4185-aa10-fc6b3ca873fd`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:14,088::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:35:14,088::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:14,088::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2722f64-6567-4185-aa10-fc6b3ca873fd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:14,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2722f64-6567-4185-aa10-fc6b3ca873fd`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:14,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:14,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:14,089::task::993::Storage.TaskManager.Task::(_decref) Task=`b2722f64-6567-4185-aa10-fc6b3ca873fd`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:35:14,143::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:35:14,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9a8ce82-7d85-4565-99d2-dec4dc9d6b47`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:35:14,143::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:35:14,143::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`898ab013-3806-4b08-a1f6-a000f7c03095`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:35:14,143::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:35:14,144::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:35:14,144::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`898ab013-3806-4b08-a1f6-a000f7c03095`::Granted request >Thread-46::DEBUG::2014-08-22 09:35:14,144::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d9a8ce82-7d85-4565-99d2-dec4dc9d6b47`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:35:14,144::task::993::Storage.TaskManager.Task::(_decref) Task=`d9a8ce82-7d85-4565-99d2-dec4dc9d6b47`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:35:14,147::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:14,147::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9a8ce82-7d85-4565-99d2-dec4dc9d6b47`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:14,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9a8ce82-7d85-4565-99d2-dec4dc9d6b47`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:35:14,147::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:35:14,147::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:35:14,147::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:35:14,147::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:35:14,148::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:35:14,148::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:35:14,148::task::993::Storage.TaskManager.Task::(_decref) Task=`d9a8ce82-7d85-4565-99d2-dec4dc9d6b47`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:14,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:14,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117578 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:16,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:16,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117992 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:17,487::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:17,487::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:17,487::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:17,487::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:17,487::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:17,488::__init__::225::IOProcess::(_processLogs) (848) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:17,488::__init__::225::IOProcess::(_processLogs) (848) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:17,488::__init__::225::IOProcess::(_processLogs) (848) Queuing response >Thread-21::DEBUG::2014-08-22 09:35:17,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:17,489::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:17,495::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:35:17,496::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000233841 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) (849) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:17,497::__init__::225::IOProcess::(_processLogs) (849) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:17,498::__init__::225::IOProcess::(_processLogs) (849) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:17,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:17,499::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) (850) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:18,099::__init__::225::IOProcess::(_processLogs) (850) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:18,100::__init__::225::IOProcess::(_processLogs) (850) Queuing response >Thread-22::DEBUG::2014-08-22 09:35:18,101::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:18,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:18,107::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:35:18,108::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214651 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:18,108::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:18,108::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:18,109::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:18,109::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:18,109::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:18,109::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:18,109::__init__::225::IOProcess::(_processLogs) (851) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:18,109::__init__::225::IOProcess::(_processLogs) (851) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:18,110::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(851) Building response >Thread-17::DEBUG::2014-08-22 09:35:18,110::__init__::225::IOProcess::(_processLogs) (851) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:18,110::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:18,110::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:18,111::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:18,111::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:35:18,111::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:18,111::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) (852) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) (852) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) (852) Building response >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) (852) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:18,112::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:35:18,113::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:18,113::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:18,113::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:18,113::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:18,113::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:18,113::__init__::225::IOProcess::(_processLogs) (853) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) (853) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) (853) Building response >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) (853) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:35:18,114::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:18,115::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:18,115::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) (854) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) (854) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) (854) Building response >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) (854) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:18,116::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:35:18,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:18,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117607 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:20,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:20,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011698 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:22,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:22,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118764 s, 86.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:24,192::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:35:24,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`456fe28c-89f2-4317-8a6d-6611e7a74c89`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:24,193::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:35:24,193::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:24,193::task::1191::Storage.TaskManager.Task::(prepare) Task=`456fe28c-89f2-4317-8a6d-6611e7a74c89`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:24,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`456fe28c-89f2-4317-8a6d-6611e7a74c89`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:24,194::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:24,194::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:24,194::task::993::Storage.TaskManager.Task::(_decref) Task=`456fe28c-89f2-4317-8a6d-6611e7a74c89`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:35:24,234::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:35:24,234::task::595::Storage.TaskManager.Task::(_updateState) Task=`32b77d66-96bf-453b-94c7-dd622c44e60c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:35:24,234::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:35:24,234::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`466bace3-b403-4a86-8c1f-a52106f9ef6d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:35:24,234::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:35:24,235::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:35:24,235::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`466bace3-b403-4a86-8c1f-a52106f9ef6d`::Granted request >Thread-46::DEBUG::2014-08-22 09:35:24,235::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`32b77d66-96bf-453b-94c7-dd622c44e60c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:35:24,235::task::993::Storage.TaskManager.Task::(_decref) Task=`32b77d66-96bf-453b-94c7-dd622c44e60c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:35:24,237::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:24,237::task::1191::Storage.TaskManager.Task::(prepare) Task=`32b77d66-96bf-453b-94c7-dd622c44e60c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:24,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`32b77d66-96bf-453b-94c7-dd622c44e60c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:35:24,237::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:35:24,237::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:35:24,237::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:35:24,238::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:35:24,238::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:35:24,238::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:35:24,238::task::993::Storage.TaskManager.Task::(_decref) Task=`32b77d66-96bf-453b-94c7-dd622c44e60c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:24,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:24,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174942 s, 58.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:25,217::task::595::Storage.TaskManager.Task::(_updateState) Task=`7452a194-40e6-4991-a567-230b875843d9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:25,217::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:35:25,217::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233841', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214651', 'lastCheck': '7.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:25,217::task::1191::Storage.TaskManager.Task::(prepare) Task=`7452a194-40e6-4991-a567-230b875843d9`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000233841', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214651', 'lastCheck': '7.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:25,217::task::595::Storage.TaskManager.Task::(_updateState) Task=`7452a194-40e6-4991-a567-230b875843d9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:25,217::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:25,217::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:25,217::task::993::Storage.TaskManager.Task::(_decref) Task=`7452a194-40e6-4991-a567-230b875843d9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:26,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:26,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012017 s, 85.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:27,499::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) (855) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:27,500::__init__::225::IOProcess::(_processLogs) (855) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:27,501::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(855) Building response >Thread-17::DEBUG::2014-08-22 09:35:27,502::__init__::225::IOProcess::(_processLogs) (855) Queuing response >Thread-21::DEBUG::2014-08-22 09:35:27,502::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:27,502::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:27,509::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:35:27,509::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00023808 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:27,510::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:27,510::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:27,510::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:27,511::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) (856) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) (856) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) (856) Building response >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) (856) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:27,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:27,513::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:28,116::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:28,116::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:28,116::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:28,117::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:35:28,117::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:28,117::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:28,123::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:28,124::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:28,124::__init__::225::IOProcess::(_processLogs) (857) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:35:28,125::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000527612 s, 1.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:28,125::__init__::225::IOProcess::(_processLogs) (857) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:28,126::__init__::225::IOProcess::(_processLogs) (857) Building response >Thread-17::DEBUG::2014-08-22 09:35:28,126::__init__::225::IOProcess::(_processLogs) (857) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:28,126::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:28,126::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:28,126::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:28,126::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) (858) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) (858) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:28,127::__init__::225::IOProcess::(_processLogs) (858) Building response >Thread-17::DEBUG::2014-08-22 09:35:28,128::__init__::225::IOProcess::(_processLogs) (858) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:28,128::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:28,129::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:28,129::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:35:28,129::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:28,129::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) (859) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) (859) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) (859) Building response >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) (859) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:35:28,130::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:28,131::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:28,131::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:28,131::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:28,131::__init__::225::IOProcess::(_processLogs) (860) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:28,131::__init__::225::IOProcess::(_processLogs) (860) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:28,131::__init__::225::IOProcess::(_processLogs) (860) Building response >Thread-17::DEBUG::2014-08-22 09:35:28,132::__init__::225::IOProcess::(_processLogs) (860) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:28,132::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:28,132::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:28,132::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:35:28,132::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:28,132::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:28,133::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) (861) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) (861) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) (861) Building response >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) (861) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:28,134::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:35:28,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:28,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119681 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:30,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:30,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116434 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:32,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:32,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021442 s, 47.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:34,362::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:35:34,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`0eca67c5-a6cb-4a91-9c8c-a92160025aba`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:34,363::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:35:34,363::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:34,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`0eca67c5-a6cb-4a91-9c8c-a92160025aba`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:34,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`0eca67c5-a6cb-4a91-9c8c-a92160025aba`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:34,363::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:34,363::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:34,364::task::993::Storage.TaskManager.Task::(_decref) Task=`0eca67c5-a6cb-4a91-9c8c-a92160025aba`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:35:34,405::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:35:34,405::task::595::Storage.TaskManager.Task::(_updateState) Task=`99b5d613-721f-4b56-a8db-60750534831a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:35:34,405::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:35:34,405::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b911fcce-1840-4df1-af6a-7160558c9928`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:35:34,406::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:35:34,406::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:35:34,406::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b911fcce-1840-4df1-af6a-7160558c9928`::Granted request >Thread-46::DEBUG::2014-08-22 09:35:34,406::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`99b5d613-721f-4b56-a8db-60750534831a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:35:34,406::task::993::Storage.TaskManager.Task::(_decref) Task=`99b5d613-721f-4b56-a8db-60750534831a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:35:34,408::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:34,408::task::1191::Storage.TaskManager.Task::(prepare) Task=`99b5d613-721f-4b56-a8db-60750534831a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:34,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`99b5d613-721f-4b56-a8db-60750534831a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:35:34,408::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:35:34,408::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:35:34,409::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:35:34,409::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:35:34,409::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:35:34,409::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:35:34,409::task::993::Storage.TaskManager.Task::(_decref) Task=`99b5d613-721f-4b56-a8db-60750534831a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:34,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:34,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115982 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:36,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:36,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118152 s, 86.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:37,512::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:37,513::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:37,513::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:37,513::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:37,513::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:37,513::__init__::225::IOProcess::(_processLogs) (862) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:37,513::__init__::225::IOProcess::(_processLogs) (862) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:37,514::__init__::225::IOProcess::(_processLogs) (862) Queuing response >Thread-21::DEBUG::2014-08-22 09:35:37,514::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:37,514::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:37,521::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:35:37,522::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000210387 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:37,522::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:37,522::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:37,522::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:37,522::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:37,522::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:37,523::__init__::225::IOProcess::(_processLogs) (863) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:37,523::__init__::225::IOProcess::(_processLogs) (863) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:37,523::__init__::225::IOProcess::(_processLogs) (863) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:37,524::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:37,524::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:38,134::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:38,134::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:38,134::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:38,135::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:38,135::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:38,135::__init__::225::IOProcess::(_processLogs) (864) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:38,135::__init__::225::IOProcess::(_processLogs) (864) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:38,135::__init__::225::IOProcess::(_processLogs) (864) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:38,136::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:35:38,136::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:38,136::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:35:38,142::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000220058 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) (865) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:38,143::__init__::225::IOProcess::(_processLogs) (865) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:38,144::__init__::225::IOProcess::(_processLogs) DEBUG|(865) Building response >Thread-17::DEBUG::2014-08-22 09:35:38,144::__init__::225::IOProcess::(_processLogs) (865) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:38,145::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:38,145::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:38,145::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:38,145::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:35:38,145::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:38,145::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) (866) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) (866) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) (866) Building response >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) (866) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:38,146::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:38,147::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:38,147::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:35:38,147::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:38,147::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:38,147::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:38,147::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) (867) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) (867) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) (867) Building response >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) (867) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:35:38,148::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:38,149::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:38,149::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) (868) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) (868) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) (868) Building response >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) (868) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:38,150::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:35:38,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:38,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119839 s, 85.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:40,450::task::595::Storage.TaskManager.Task::(_updateState) Task=`67769b2e-db61-4a56-b0b6-003d75399644`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:40,450::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:35:40,450::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210387', 'lastCheck': '2.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000220058', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:40,450::task::1191::Storage.TaskManager.Task::(prepare) Task=`67769b2e-db61-4a56-b0b6-003d75399644`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210387', 'lastCheck': '2.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000220058', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:40,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`67769b2e-db61-4a56-b0b6-003d75399644`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:40,451::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:40,451::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:40,451::task::993::Storage.TaskManager.Task::(_decref) Task=`67769b2e-db61-4a56-b0b6-003d75399644`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:41,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:41,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01156 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:43,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:43,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118336 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:44,479::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:35:44,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`9952f21a-1148-4da0-958a-ba0c2e156c68`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:44,480::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:35:44,480::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:44,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`9952f21a-1148-4da0-958a-ba0c2e156c68`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:44,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`9952f21a-1148-4da0-958a-ba0c2e156c68`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:44,480::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:44,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:44,480::task::993::Storage.TaskManager.Task::(_decref) Task=`9952f21a-1148-4da0-958a-ba0c2e156c68`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:35:44,510::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:35:44,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`68688ec0-c92a-42ce-b65d-b46f2b6d711c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:35:44,510::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:35:44,510::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`68d67761-856e-4547-a825-dd94e9ee6dc2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:35:44,510::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:35:44,510::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:35:44,511::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`68d67761-856e-4547-a825-dd94e9ee6dc2`::Granted request >Thread-46::DEBUG::2014-08-22 09:35:44,511::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`68688ec0-c92a-42ce-b65d-b46f2b6d711c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:35:44,511::task::993::Storage.TaskManager.Task::(_decref) Task=`68688ec0-c92a-42ce-b65d-b46f2b6d711c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:35:44,513::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:44,513::task::1191::Storage.TaskManager.Task::(prepare) Task=`68688ec0-c92a-42ce-b65d-b46f2b6d711c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:44,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`68688ec0-c92a-42ce-b65d-b46f2b6d711c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:35:44,513::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:35:44,513::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:35:44,513::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:35:44,513::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:35:44,513::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:35:44,514::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:35:44,514::task::993::Storage.TaskManager.Task::(_decref) Task=`68688ec0-c92a-42ce-b65d-b46f2b6d711c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:45,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:45,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011638 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:47,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:47,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119453 s, 85.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) (869) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:47,525::__init__::225::IOProcess::(_processLogs) (869) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:47,526::__init__::225::IOProcess::(_processLogs) DEBUG|(869) Building response >Thread-21::DEBUG::2014-08-22 09:35:47,527::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:47,527::__init__::225::IOProcess::(_processLogs) (869) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:47,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:35:47,534::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.0002146 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:47,534::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:47,535::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:47,535::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) (870) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) (870) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) (870) Building response >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) (870) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:47,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:47,537::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:48,150::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:48,150::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:48,150::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:48,151::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:35:48,152::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:48,152::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:48,158::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:35:48,159::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0002475 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:48,159::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:48,160::__init__::225::IOProcess::(_processLogs) (871) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:48,160::__init__::225::IOProcess::(_processLogs) (871) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:48,160::__init__::225::IOProcess::(_processLogs) (871) Building response >Thread-17::DEBUG::2014-08-22 09:35:48,160::__init__::225::IOProcess::(_processLogs) (871) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:48,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:48,160::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) (872) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:48,161::__init__::225::IOProcess::(_processLogs) (872) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:48,162::__init__::225::IOProcess::(_processLogs) DEBUG|(872) Building response >Thread-17::DEBUG::2014-08-22 09:35:48,162::__init__::225::IOProcess::(_processLogs) (872) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:48,163::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:48,163::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:48,163::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:48,163::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:35:48,163::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:48,163::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) (873) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) (873) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) (873) Building response >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) (873) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:48,164::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:48,165::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) (874) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) (874) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) (874) Building response >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) (874) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:35:48,166::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:48,167::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:48,167::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:48,167::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:48,167::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:48,167::__init__::225::IOProcess::(_processLogs) (875) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:48,168::__init__::225::IOProcess::(_processLogs) (875) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:48,168::__init__::225::IOProcess::(_processLogs) (875) Building response >Thread-17::DEBUG::2014-08-22 09:35:48,168::__init__::225::IOProcess::(_processLogs) (875) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:48,168::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:48,168::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:35:49,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:49,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0144207 s, 71.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:51,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:51,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012184 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:35:53,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:53,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117687 s, 87.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:54,560::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:35:54,561::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd26d515-1de4-45d2-b49f-34e5b6703651`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:54,561::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:35:54,561::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:54,562::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd26d515-1de4-45d2-b49f-34e5b6703651`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:35:54,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd26d515-1de4-45d2-b49f-34e5b6703651`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:54,562::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:54,562::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:54,562::task::993::Storage.TaskManager.Task::(_decref) Task=`bd26d515-1de4-45d2-b49f-34e5b6703651`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:35:54,605::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:35:54,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e65a47b-1c9a-44c8-83d8-3d0e1850756c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:35:54,605::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:35:54,606::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6b6fb850-6327-430b-ac5c-fbd76ac6de91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:35:54,606::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:35:54,606::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:35:54,606::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6b6fb850-6327-430b-ac5c-fbd76ac6de91`::Granted request >Thread-46::DEBUG::2014-08-22 09:35:54,606::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7e65a47b-1c9a-44c8-83d8-3d0e1850756c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:35:54,606::task::993::Storage.TaskManager.Task::(_decref) Task=`7e65a47b-1c9a-44c8-83d8-3d0e1850756c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:35:54,608::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:54,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e65a47b-1c9a-44c8-83d8-3d0e1850756c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:35:54,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e65a47b-1c9a-44c8-83d8-3d0e1850756c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:35:54,608::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:35:54,608::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:35:54,609::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:35:54,609::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:35:54,609::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:35:54,609::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:35:54,609::task::993::Storage.TaskManager.Task::(_decref) Task=`7e65a47b-1c9a-44c8-83d8-3d0e1850756c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:55,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:55,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117212 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:35:55,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad78bfef-89cc-47ca-8ff9-5739e5064803`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:35:55,685::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:35:55,685::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0002146', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002475', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:55,685::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad78bfef-89cc-47ca-8ff9-5739e5064803`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0002146', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002475', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:35:55,685::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad78bfef-89cc-47ca-8ff9-5739e5064803`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:35:55,685::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:35:55,685::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:35:55,685::task::993::Storage.TaskManager.Task::(_decref) Task=`ad78bfef-89cc-47ca-8ff9-5739e5064803`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:35:57,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:57,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120889 s, 84.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:57,538::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) (876) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:57,539::__init__::225::IOProcess::(_processLogs) (876) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:57,540::__init__::225::IOProcess::(_processLogs) (876) Queuing response >Thread-21::DEBUG::2014-08-22 09:35:57,540::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:57,541::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:57,546::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:35:57,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000191099 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:57,548::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:35:57,548::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:57,548::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:57,548::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:57,548::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:57,548::__init__::225::IOProcess::(_processLogs) (877) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:57,549::__init__::225::IOProcess::(_processLogs) (877) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:57,549::__init__::225::IOProcess::(_processLogs) (877) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:57,550::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:57,550::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) (878) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:58,168::__init__::225::IOProcess::(_processLogs) (878) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:58,169::__init__::225::IOProcess::(_processLogs) (878) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:58,169::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:35:58,169::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:35:58,170::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:35:58,175::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000260129 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) (879) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:35:58,176::__init__::225::IOProcess::(_processLogs) (879) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:35:58,177::__init__::225::IOProcess::(_processLogs) (879) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:58,178::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:58,178::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:35:58,178::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:35:58,178::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) (880) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) (880) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) (880) Building response >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) (880) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:58,179::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) (881) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) (881) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) (881) Building response >Thread-17::DEBUG::2014-08-22 09:35:58,180::__init__::225::IOProcess::(_processLogs) (881) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:35:58,181::__init__::225::IOProcess::(_processLogs) (882) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:35:58,182::__init__::225::IOProcess::(_processLogs) (882) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:35:58,182::__init__::225::IOProcess::(_processLogs) (882) Queuing response >Thread-17::DEBUG::2014-08-22 09:35:58,182::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:35:58,183::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:35:59,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:35:59,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117418 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:01,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:01,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115927 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:03,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:03,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01161 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:04,656::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-47::DEBUG::2014-08-22 09:36:04,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8ad5059-395e-429d-8da0-c1be00b442a0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:04,657::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:36:04,657::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:04,657::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8ad5059-395e-429d-8da0-c1be00b442a0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:04,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8ad5059-395e-429d-8da0-c1be00b442a0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:04,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:04,658::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:04,658::task::993::Storage.TaskManager.Task::(_decref) Task=`c8ad5059-395e-429d-8da0-c1be00b442a0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:36:04,682::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-46::DEBUG::2014-08-22 09:36:04,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`2607bfe1-c971-4979-8ecd-62aebc6b0dc0`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:36:04,683::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:36:04,683::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`02c37501-a00d-43d8-b3a7-0232c0c6e653`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:36:04,683::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:36:04,683::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:36:04,683::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`02c37501-a00d-43d8-b3a7-0232c0c6e653`::Granted request >Thread-46::DEBUG::2014-08-22 09:36:04,684::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2607bfe1-c971-4979-8ecd-62aebc6b0dc0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:36:04,684::task::993::Storage.TaskManager.Task::(_decref) Task=`2607bfe1-c971-4979-8ecd-62aebc6b0dc0`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:36:04,686::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:04,686::task::1191::Storage.TaskManager.Task::(prepare) Task=`2607bfe1-c971-4979-8ecd-62aebc6b0dc0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:04,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`2607bfe1-c971-4979-8ecd-62aebc6b0dc0`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:36:04,686::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:36:04,686::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:36:04,686::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:36:04,686::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:36:04,686::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:36:04,686::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:36:04,687::task::993::Storage.TaskManager.Task::(_decref) Task=`2607bfe1-c971-4979-8ecd-62aebc6b0dc0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:05,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:05,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119242 s, 85.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:07,550::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:07,551::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:07,551::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:07,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:07,551::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:07,551::__init__::225::IOProcess::(_processLogs) (883) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:07,551::__init__::225::IOProcess::(_processLogs) (883) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:07,552::__init__::225::IOProcess::(_processLogs) (883) Queuing response >Thread-21::DEBUG::2014-08-22 09:36:07,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:07,552::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:07,559::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:36:07,560::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000202152 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:07,560::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:07,560::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:07,560::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:07,560::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:07,561::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:07,561::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:07,561::__init__::225::IOProcess::(_processLogs) (884) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:07,561::__init__::225::IOProcess::(_processLogs) (884) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:07,561::__init__::225::IOProcess::(_processLogs) (884) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:07,562::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:07,562::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:36:07,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:07,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117289 s, 87.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:08,183::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:08,183::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:08,183::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:08,184::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:08,184::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:08,184::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:08,184::__init__::225::IOProcess::(_processLogs) (885) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:08,184::__init__::225::IOProcess::(_processLogs) (885) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:08,184::__init__::225::IOProcess::(_processLogs) DEBUG|(885) Building response >Thread-22::DEBUG::2014-08-22 09:36:08,185::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:08,185::__init__::225::IOProcess::(_processLogs) (885) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:08,191::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:36:08,192::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000288084 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:08,193::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:08,193::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) (886) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:08,194::__init__::225::IOProcess::(_processLogs) (886) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:08,195::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(886) Building response >Thread-17::DEBUG::2014-08-22 09:36:08,195::__init__::225::IOProcess::(_processLogs) (886) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:08,196::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:08,196::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:08,196::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:08,196::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:36:08,196::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:08,196::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) (887) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) (887) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) (887) Building response >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) (887) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:08,197::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:08,198::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:08,198::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:36:08,198::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:08,198::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:08,198::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) (888) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) (888) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) (888) Building response >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) (888) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:36:08,199::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:08,200::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:08,200::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) (889) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) (889) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) (889) Building response >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) (889) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:08,201::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:36:09,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:09,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0146956 s, 69.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:10,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`d40c63e6-be49-4d7a-9f41-cbd9dff7acb4`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:10,936::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:36:10,936::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000202152', 'lastCheck': '3.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000288084', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:10,936::task::1191::Storage.TaskManager.Task::(prepare) Task=`d40c63e6-be49-4d7a-9f41-cbd9dff7acb4`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000202152', 'lastCheck': '3.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000288084', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:10,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`d40c63e6-be49-4d7a-9f41-cbd9dff7acb4`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:10,936::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:10,936::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:10,936::task::993::Storage.TaskManager.Task::(_decref) Task=`d40c63e6-be49-4d7a-9f41-cbd9dff7acb4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:11,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:11,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115279 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:13,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:13,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116117 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:14,778::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:36:14,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`25142761-53c1-46c4-a85d-21276966cef2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:14,779::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:36:14,779::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:14,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`25142761-53c1-46c4-a85d-21276966cef2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:14,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`25142761-53c1-46c4-a85d-21276966cef2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:14,780::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:14,780::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:14,780::task::993::Storage.TaskManager.Task::(_decref) Task=`25142761-53c1-46c4-a85d-21276966cef2`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:36:14,812::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:36:14,812::task::595::Storage.TaskManager.Task::(_updateState) Task=`396bfc1f-9aee-4a89-abcf-52b801b6326b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:36:14,812::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:36:14,812::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12a52a19-75cd-48c5-8444-44804ce621cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:36:14,813::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:36:14,813::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:36:14,813::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12a52a19-75cd-48c5-8444-44804ce621cb`::Granted request >Thread-46::DEBUG::2014-08-22 09:36:14,813::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`396bfc1f-9aee-4a89-abcf-52b801b6326b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:36:14,813::task::993::Storage.TaskManager.Task::(_decref) Task=`396bfc1f-9aee-4a89-abcf-52b801b6326b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:36:14,815::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:14,815::task::1191::Storage.TaskManager.Task::(prepare) Task=`396bfc1f-9aee-4a89-abcf-52b801b6326b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:14,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`396bfc1f-9aee-4a89-abcf-52b801b6326b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:36:14,815::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:36:14,815::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:36:14,816::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:36:14,816::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:36:14,816::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:36:14,816::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:36:14,816::task::993::Storage.TaskManager.Task::(_decref) Task=`396bfc1f-9aee-4a89-abcf-52b801b6326b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:15,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:15,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114811 s, 89.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) (890) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:17,563::__init__::225::IOProcess::(_processLogs) (890) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:17,564::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(890) Building response >Thread-21::DEBUG::2014-08-22 09:36:17,565::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:17,565::__init__::225::IOProcess::(_processLogs) (890) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:17,571::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:36:17,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000186804 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:17,572::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:17,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:17,573::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:17,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:17,574::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:17,574::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:17,574::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:17,574::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:17,574::__init__::225::IOProcess::(_processLogs) (891) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:17,574::__init__::225::IOProcess::(_processLogs) (891) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:17,575::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(891) Building response >Thread-17::DEBUG::2014-08-22 09:36:17,575::__init__::225::IOProcess::(_processLogs) (891) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:17,576::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:17,576::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:36:17,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:17,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119158 s, 85.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:18,201::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:18,201::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:18,201::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:18,201::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:18,201::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:18,202::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:18,202::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:18,202::__init__::225::IOProcess::(_processLogs) (892) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:18,202::__init__::225::IOProcess::(_processLogs) (892) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:18,202::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(892) Building response >Thread-17::DEBUG::2014-08-22 09:36:18,203::__init__::225::IOProcess::(_processLogs) (892) Queuing response >Thread-22::DEBUG::2014-08-22 09:36:18,203::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:18,204::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:18,210::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:36:18,211::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000193403 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) (893) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) (893) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:18,212::__init__::225::IOProcess::(_processLogs) (893) Building response >Thread-17::DEBUG::2014-08-22 09:36:18,213::__init__::225::IOProcess::(_processLogs) (893) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:18,213::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:18,213::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:18,214::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:18,214::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:36:18,214::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:18,214::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:18,214::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:18,214::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:18,215::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:18,215::__init__::225::IOProcess::(_processLogs) (894) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:18,215::__init__::225::IOProcess::(_processLogs) (894) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:18,215::__init__::225::IOProcess::(_processLogs) (894) Building response >Thread-17::DEBUG::2014-08-22 09:36:18,215::__init__::225::IOProcess::(_processLogs) (894) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:18,215::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:36:18,216::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:18,216::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:18,216::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:36:18,216::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:18,216::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:18,217::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:18,217::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:18,217::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:18,217::__init__::225::IOProcess::(_processLogs) (895) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) (895) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) (895) Building response >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) (895) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:36:18,218::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:18,219::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:18,219::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:18,219::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:18,219::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:18,219::__init__::225::IOProcess::(_processLogs) (896) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:18,220::__init__::225::IOProcess::(_processLogs) (896) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:18,220::__init__::225::IOProcess::(_processLogs) (896) Building response >Thread-17::DEBUG::2014-08-22 09:36:18,220::__init__::225::IOProcess::(_processLogs) (896) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:18,220::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:18,220::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:36:19,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:20,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117804 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:22,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:22,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011734 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:24,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:24,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116228 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:24,874::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:36:24,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`38ba5b2c-0a03-4791-bd82-70d80a50bb45`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:24,874::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:36:24,874::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:24,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`38ba5b2c-0a03-4791-bd82-70d80a50bb45`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:24,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`38ba5b2c-0a03-4791-bd82-70d80a50bb45`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:24,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:24,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:24,875::task::993::Storage.TaskManager.Task::(_decref) Task=`38ba5b2c-0a03-4791-bd82-70d80a50bb45`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:36:24,921::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:36:24,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`1bb2cac8-114d-45e7-9ef7-595fb90a14b9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:36:24,921::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:36:24,922::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`350b958e-4fa7-4782-a129-305510dbd2c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:36:24,922::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:36:24,922::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:36:24,922::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`350b958e-4fa7-4782-a129-305510dbd2c6`::Granted request >Thread-46::DEBUG::2014-08-22 09:36:24,922::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1bb2cac8-114d-45e7-9ef7-595fb90a14b9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:36:24,922::task::993::Storage.TaskManager.Task::(_decref) Task=`1bb2cac8-114d-45e7-9ef7-595fb90a14b9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:36:24,924::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:24,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`1bb2cac8-114d-45e7-9ef7-595fb90a14b9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:24,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`1bb2cac8-114d-45e7-9ef7-595fb90a14b9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:36:24,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:36:24,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:36:24,925::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:36:24,925::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:36:24,925::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:36:24,925::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:36:24,925::task::993::Storage.TaskManager.Task::(_decref) Task=`1bb2cac8-114d-45e7-9ef7-595fb90a14b9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:26,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:26,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117173 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:26,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cf218fa-c2fb-4ada-bd43-9eb720c46d7b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:26,214::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:36:26,214::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000186804', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000193403', 'lastCheck': '8.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:26,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`4cf218fa-c2fb-4ada-bd43-9eb720c46d7b`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000186804', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000193403', 'lastCheck': '8.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:26,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cf218fa-c2fb-4ada-bd43-9eb720c46d7b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:26,215::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:26,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:26,215::task::993::Storage.TaskManager.Task::(_decref) Task=`4cf218fa-c2fb-4ada-bd43-9eb720c46d7b`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:27,576::__init__::225::IOProcess::(_processLogs) (897) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:27,577::__init__::225::IOProcess::(_processLogs) (897) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:27,577::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(897) Building response >Thread-21::DEBUG::2014-08-22 09:36:27,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:27,578::__init__::225::IOProcess::(_processLogs) (897) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:27,584::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:36:27,585::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000193229 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:27,585::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) (898) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:27,587::__init__::225::IOProcess::(_processLogs) (898) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:27,588::__init__::225::IOProcess::(_processLogs) (898) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:27,589::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:27,589::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:36:28,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:28,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116275 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:28,222::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:28,223::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:28,224::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:28,224::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:28,225::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:28,226::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:28,226::__init__::225::IOProcess::(_processLogs) (899) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:28,227::__init__::225::IOProcess::(_processLogs) (899) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:28,230::__init__::225::IOProcess::(_processLogs) (899) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:28,231::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:36:28,231::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:28,237::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:36:28,239::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000188145 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:28,240::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:28,241::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:28,243::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:28,244::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:28,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:28,245::__init__::225::IOProcess::(_processLogs) (900) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:28,245::__init__::225::IOProcess::(_processLogs) (900) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:28,246::__init__::225::IOProcess::(_processLogs) (900) Building response >Thread-17::DEBUG::2014-08-22 09:36:28,246::__init__::225::IOProcess::(_processLogs) (900) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:28,246::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:28,247::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:28,248::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:36:28,248::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:28,249::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:28,250::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:28,250::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:28,251::__init__::225::IOProcess::(_processLogs) (901) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:28,251::__init__::225::IOProcess::(_processLogs) (901) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:28,251::__init__::225::IOProcess::(_processLogs) (901) Building response >Thread-17::DEBUG::2014-08-22 09:36:28,252::__init__::225::IOProcess::(_processLogs) (901) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:28,252::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:28,253::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:28,253::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:36:28,254::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:28,255::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:28,256::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:28,256::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:28,256::__init__::225::IOProcess::(_processLogs) (902) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:28,257::__init__::225::IOProcess::(_processLogs) (902) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:28,257::__init__::225::IOProcess::(_processLogs) (902) Building response >Thread-17::DEBUG::2014-08-22 09:36:28,258::__init__::225::IOProcess::(_processLogs) (902) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:28,258::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:28,258::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:28,259::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:36:28,259::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:28,260::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:28,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:28,262::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:28,262::__init__::225::IOProcess::(_processLogs) (903) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:28,263::__init__::225::IOProcess::(_processLogs) (903) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:28,263::__init__::225::IOProcess::(_processLogs) (903) Building response >Thread-17::DEBUG::2014-08-22 09:36:28,263::__init__::225::IOProcess::(_processLogs) (903) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:28,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:28,264::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:36:30,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:30,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117131 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:32,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:32,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184986 s, 55.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:34,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:34,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011698 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:34,968::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:36:34,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`73a32e6e-4588-4d9a-9e1a-b0c472552b1d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:34,969::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:36:34,969::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:34,969::task::1191::Storage.TaskManager.Task::(prepare) Task=`73a32e6e-4588-4d9a-9e1a-b0c472552b1d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:34,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`73a32e6e-4588-4d9a-9e1a-b0c472552b1d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:34,970::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:34,970::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:34,970::task::993::Storage.TaskManager.Task::(_decref) Task=`73a32e6e-4588-4d9a-9e1a-b0c472552b1d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:36:35,047::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:36:35,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`46b636f2-1a1a-49b2-9a62-1fcad933846e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:36:35,048::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:36:35,048::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3675714d-f958-4d7d-95f2-806a89c472c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:36:35,048::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:36:35,048::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:36:35,048::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3675714d-f958-4d7d-95f2-806a89c472c6`::Granted request >Thread-46::DEBUG::2014-08-22 09:36:35,048::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`46b636f2-1a1a-49b2-9a62-1fcad933846e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:36:35,049::task::993::Storage.TaskManager.Task::(_decref) Task=`46b636f2-1a1a-49b2-9a62-1fcad933846e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:36:35,051::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:35,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`46b636f2-1a1a-49b2-9a62-1fcad933846e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:35,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`46b636f2-1a1a-49b2-9a62-1fcad933846e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:36:35,051::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:36:35,051::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:36:35,051::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:36:35,051::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:36:35,051::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:36:35,051::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:36:35,051::task::993::Storage.TaskManager.Task::(_decref) Task=`46b636f2-1a1a-49b2-9a62-1fcad933846e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:36,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:36,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118708 s, 86.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:37,589::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:37,589::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:37,590::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:37,591::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:37,591::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:37,591::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:37,591::__init__::225::IOProcess::(_processLogs) (904) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:37,591::__init__::225::IOProcess::(_processLogs) (904) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:37,591::__init__::225::IOProcess::(_processLogs) (904) Queuing response >Thread-21::DEBUG::2014-08-22 09:36:37,592::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:37,592::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:37,598::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:36:37,599::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00019937 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) (905) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:37,600::__init__::225::IOProcess::(_processLogs) (905) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:37,601::__init__::225::IOProcess::(_processLogs) (905) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:37,602::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:37,602::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:38,262::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:38,262::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:38,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:38,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:38,262::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:38,262::__init__::225::IOProcess::(_processLogs) (906) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:38,263::__init__::225::IOProcess::(_processLogs) (906) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:38,263::__init__::225::IOProcess::(_processLogs) (906) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:38,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:36:38,263::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:38,264::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:36:38,270::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000220837 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:38,270::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:38,270::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:38,271::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:38,271::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:38,271::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:38,271::__init__::225::IOProcess::(_processLogs) (907) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:38,271::__init__::225::IOProcess::(_processLogs) (907) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:38,272::__init__::225::IOProcess::(_processLogs) (907) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:38,272::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:38,272::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:38,272::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:36:38,273::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:38,273::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:38,273::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:38,273::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:38,273::__init__::225::IOProcess::(_processLogs) (908) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) (908) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) (908) Building response >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) (908) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:36:38,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) (909) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) (909) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) (909) Building response >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) (909) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:38,275::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:38,276::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:36:38,276::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:38,276::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:38,276::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) (910) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) (910) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) (910) Building response >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) (910) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:38,277::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:36:38,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:38,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120148 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:40,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:40,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012063 s, 84.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:41,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`bba95cf1-7558-48d8-9720-0c8ad2652448`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:41,471::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:36:41,471::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00019937', 'lastCheck': '3.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000220837', 'lastCheck': '3.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:41,471::task::1191::Storage.TaskManager.Task::(prepare) Task=`bba95cf1-7558-48d8-9720-0c8ad2652448`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00019937', 'lastCheck': '3.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000220837', 'lastCheck': '3.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:41,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`bba95cf1-7558-48d8-9720-0c8ad2652448`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:41,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:41,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:41,472::task::993::Storage.TaskManager.Task::(_decref) Task=`bba95cf1-7558-48d8-9720-0c8ad2652448`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:42,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:42,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122606 s, 83.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:44,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:44,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189195 s, 54.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:45,114::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:36:45,114::task::595::Storage.TaskManager.Task::(_updateState) Task=`eae03872-df88-4a3e-bf23-a36e5868cdbd`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:45,115::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:36:45,115::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:45,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`eae03872-df88-4a3e-bf23-a36e5868cdbd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:45,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`eae03872-df88-4a3e-bf23-a36e5868cdbd`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:45,115::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:45,115::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:45,115::task::993::Storage.TaskManager.Task::(_decref) Task=`eae03872-df88-4a3e-bf23-a36e5868cdbd`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:36:45,145::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:36:45,146::task::595::Storage.TaskManager.Task::(_updateState) Task=`965feff4-7f8a-4c05-8daa-40587d7f16f7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:36:45,146::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:36:45,146::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`938b8131-2150-4890-a830-87da80a8f44c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:36:45,146::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:36:45,146::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:36:45,146::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`938b8131-2150-4890-a830-87da80a8f44c`::Granted request >Thread-46::DEBUG::2014-08-22 09:36:45,147::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`965feff4-7f8a-4c05-8daa-40587d7f16f7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:36:45,147::task::993::Storage.TaskManager.Task::(_decref) Task=`965feff4-7f8a-4c05-8daa-40587d7f16f7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:36:45,149::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:45,149::task::1191::Storage.TaskManager.Task::(prepare) Task=`965feff4-7f8a-4c05-8daa-40587d7f16f7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:45,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`965feff4-7f8a-4c05-8daa-40587d7f16f7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:36:45,149::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:36:45,149::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:36:45,149::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:36:45,149::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:36:45,149::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:36:45,150::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:36:45,150::task::993::Storage.TaskManager.Task::(_decref) Task=`965feff4-7f8a-4c05-8daa-40587d7f16f7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:46,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:46,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115187 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:47,602::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:47,602::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:47,602::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:47,602::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:47,602::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:47,603::__init__::225::IOProcess::(_processLogs) (911) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:47,603::__init__::225::IOProcess::(_processLogs) (911) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:47,603::__init__::225::IOProcess::(_processLogs) (911) Queuing response >Thread-21::DEBUG::2014-08-22 09:36:47,604::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:47,604::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:47,610::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:36:47,611::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000177233 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) (912) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:47,612::__init__::225::IOProcess::(_processLogs) (912) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:47,613::__init__::225::IOProcess::(_processLogs) (912) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:47,614::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:47,614::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:48,277::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:48,277::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:48,278::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:36:48,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:48,278::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:48,284::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:36:48,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00024534 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:48,285::__init__::225::IOProcess::(_processLogs) (913) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:48,286::__init__::225::IOProcess::(_processLogs) (913) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:48,286::__init__::225::IOProcess::(_processLogs) (913) Building response >Thread-17::DEBUG::2014-08-22 09:36:48,286::__init__::225::IOProcess::(_processLogs) (913) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:48,286::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) (914) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:48,287::__init__::225::IOProcess::(_processLogs) (914) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:48,288::__init__::225::IOProcess::(_processLogs) (914) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:48,289::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:48,289::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:48,289::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:36:48,289::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:48,289::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) (915) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) (915) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) (915) Building response >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) (915) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:48,290::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:36:48,291::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:48,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:48,291::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:48,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:48,291::__init__::225::IOProcess::(_processLogs) (916) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:48,291::__init__::225::IOProcess::(_processLogs) (916) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) (916) Building response >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) (916) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:48,292::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:48,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:48,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:48,293::__init__::225::IOProcess::(_processLogs) (917) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:48,293::__init__::225::IOProcess::(_processLogs) (917) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:48,293::__init__::225::IOProcess::(_processLogs) (917) Building response >Thread-17::DEBUG::2014-08-22 09:36:48,293::__init__::225::IOProcess::(_processLogs) (917) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:48,294::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:48,294::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:36:48,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:48,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122128 s, 83.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:50,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:50,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116672 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:52,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:52,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122408 s, 83.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:36:54,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:55,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117196 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:36:55,206::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:36:55,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`b457a1e5-362b-4479-9cf8-2e3897d63601`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:55,207::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:36:55,207::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:55,207::task::1191::Storage.TaskManager.Task::(prepare) Task=`b457a1e5-362b-4479-9cf8-2e3897d63601`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:36:55,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`b457a1e5-362b-4479-9cf8-2e3897d63601`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:55,207::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:55,207::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:55,208::task::993::Storage.TaskManager.Task::(_decref) Task=`b457a1e5-362b-4479-9cf8-2e3897d63601`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:36:55,243::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:36:55,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`060ab915-3b9a-4165-b785-037b23f09bc0`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:36:55,243::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:36:55,243::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c37b7051-72f6-4582-a9c3-c41c9ba4625c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:36:55,244::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:36:55,244::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:36:55,244::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c37b7051-72f6-4582-a9c3-c41c9ba4625c`::Granted request >Thread-46::DEBUG::2014-08-22 09:36:55,244::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`060ab915-3b9a-4165-b785-037b23f09bc0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:36:55,244::task::993::Storage.TaskManager.Task::(_decref) Task=`060ab915-3b9a-4165-b785-037b23f09bc0`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:36:55,246::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:55,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`060ab915-3b9a-4165-b785-037b23f09bc0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:36:55,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`060ab915-3b9a-4165-b785-037b23f09bc0`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:36:55,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:36:55,246::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:36:55,247::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:36:55,247::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:36:55,247::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:36:55,247::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:36:55,247::task::993::Storage.TaskManager.Task::(_decref) Task=`060ab915-3b9a-4165-b785-037b23f09bc0`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:36:56,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`6195248f-0b31-4615-94f8-d44753c04f05`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:36:56,727::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:36:56,728::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000177233', 'lastCheck': '9.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00024534', 'lastCheck': '8.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:56,728::task::1191::Storage.TaskManager.Task::(prepare) Task=`6195248f-0b31-4615-94f8-d44753c04f05`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000177233', 'lastCheck': '9.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00024534', 'lastCheck': '8.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:36:56,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`6195248f-0b31-4615-94f8-d44753c04f05`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:36:56,728::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:36:56,728::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:36:56,728::task::993::Storage.TaskManager.Task::(_decref) Task=`6195248f-0b31-4615-94f8-d44753c04f05`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:36:57,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:57,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115951 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:57,614::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:57,614::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:57,614::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:57,614::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:57,615::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:57,615::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:57,615::__init__::225::IOProcess::(_processLogs) (918) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:57,615::__init__::225::IOProcess::(_processLogs) (918) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:57,615::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(918) Building response >Thread-17::DEBUG::2014-08-22 09:36:57,616::__init__::225::IOProcess::(_processLogs) (918) Queuing response >Thread-21::DEBUG::2014-08-22 09:36:57,616::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:57,617::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:57,623::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:36:57,624::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000188265 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:57,624::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:57,624::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) (919) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:57,625::__init__::225::IOProcess::(_processLogs) (919) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:57,626::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(919) Building response >Thread-17::DEBUG::2014-08-22 09:36:57,627::__init__::225::IOProcess::(_processLogs) (919) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:57,627::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:57,627::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:58,293::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) (920) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:58,294::__init__::225::IOProcess::(_processLogs) (920) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:58,295::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(920) Building response >Thread-17::DEBUG::2014-08-22 09:36:58,295::__init__::225::IOProcess::(_processLogs) (920) Queuing response >Thread-22::DEBUG::2014-08-22 09:36:58,295::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:36:58,296::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:58,302::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:36:58,302::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000204262 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:36:58,303::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) (921) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:36:58,304::__init__::225::IOProcess::(_processLogs) (921) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:36:58,305::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(921) Building response >Thread-17::DEBUG::2014-08-22 09:36:58,305::__init__::225::IOProcess::(_processLogs) (921) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:58,306::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:58,306::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:36:58,306::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:58,306::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:36:58,306::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:58,306::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) (922) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) (922) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) (922) Building response >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) (922) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:58,307::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:58,308::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:58,308::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:36:58,308::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:58,308::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:58,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:58,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) (923) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) (923) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) (923) Building response >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) (923) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:36:58,309::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) (924) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) (924) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) (924) Building response >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) (924) Queuing response >Thread-17::DEBUG::2014-08-22 09:36:58,310::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:36:58,311::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:36:59,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:36:59,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121061 s, 84.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:01,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:01,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116257 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:03,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:03,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011511 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:05,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:05,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116492 s, 87.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:05,298::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:37:05,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`2bca3fd5-ab99-43e9-9da1-4b4f556a4169`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:05,299::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:37:05,299::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:05,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`2bca3fd5-ab99-43e9-9da1-4b4f556a4169`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:05,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`2bca3fd5-ab99-43e9-9da1-4b4f556a4169`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:05,299::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:05,299::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:05,299::task::993::Storage.TaskManager.Task::(_decref) Task=`2bca3fd5-ab99-43e9-9da1-4b4f556a4169`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:37:05,351::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:37:05,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`eaac6e7e-f973-4672-85c5-f5c5aad32bd1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:37:05,352::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:37:05,352::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`84ce8d34-e7ff-44d0-bc93-c35febd75963`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:37:05,352::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:37:05,352::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:37:05,352::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`84ce8d34-e7ff-44d0-bc93-c35febd75963`::Granted request >Thread-46::DEBUG::2014-08-22 09:37:05,353::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eaac6e7e-f973-4672-85c5-f5c5aad32bd1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:37:05,353::task::993::Storage.TaskManager.Task::(_decref) Task=`eaac6e7e-f973-4672-85c5-f5c5aad32bd1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:37:05,355::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:05,355::task::1191::Storage.TaskManager.Task::(prepare) Task=`eaac6e7e-f973-4672-85c5-f5c5aad32bd1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:05,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`eaac6e7e-f973-4672-85c5-f5c5aad32bd1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:37:05,355::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:37:05,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:37:05,355::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:37:05,355::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:37:05,355::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:37:05,355::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:37:05,356::task::993::Storage.TaskManager.Task::(_decref) Task=`eaac6e7e-f973-4672-85c5-f5c5aad32bd1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:07,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:07,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115395 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:07,627::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:07,627::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:07,627::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:07,627::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:07,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:07,628::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:07,628::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:07,628::__init__::225::IOProcess::(_processLogs) (925) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:07,628::__init__::225::IOProcess::(_processLogs) (925) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:07,628::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(925) Building response >Thread-21::DEBUG::2014-08-22 09:37:07,629::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:07,629::__init__::225::IOProcess::(_processLogs) (925) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:07,635::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:07,635::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:37:07,637::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000987629 s, 402 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) (926) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:07,638::__init__::225::IOProcess::(_processLogs) (926) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:07,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(926) Building response >Thread-17::DEBUG::2014-08-22 09:37:07,639::__init__::225::IOProcess::(_processLogs) (926) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:07,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:37:07,640::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:08,311::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:08,311::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:08,311::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) (927) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) (927) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) (927) Building response >Thread-17::DEBUG::2014-08-22 09:37:08,312::__init__::225::IOProcess::(_processLogs) (927) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:08,313::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:37:08,313::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:37:08,313::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:37:08,320::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000252742 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:08,320::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:08,320::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:08,320::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:08,320::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) (928) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) (928) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) (928) Building response >Thread-17::DEBUG::2014-08-22 09:37:08,321::__init__::225::IOProcess::(_processLogs) (928) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:08,322::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:08,322::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:37:08,323::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:08,323::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:08,323::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:08,323::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:08,323::__init__::225::IOProcess::(_processLogs) (929) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:08,323::__init__::225::IOProcess::(_processLogs) (929) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) (929) Building response >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) (929) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:08,324::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) (930) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) (930) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) (930) Building response >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) (930) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:08,325::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:08,326::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:37:08,326::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:08,326::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:08,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) (931) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) (931) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) (931) Building response >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) (931) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:08,327::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:37:09,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:09,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197251 s, 51.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:11,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:11,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011695 s, 87.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:11,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`a12b7afb-03ab-4ebe-8038-98a1426b41ac`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:11,893::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:37:11,893::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000987629', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000252742', 'lastCheck': '3.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:11,894::task::1191::Storage.TaskManager.Task::(prepare) Task=`a12b7afb-03ab-4ebe-8038-98a1426b41ac`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000987629', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000252742', 'lastCheck': '3.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:11,894::task::595::Storage.TaskManager.Task::(_updateState) Task=`a12b7afb-03ab-4ebe-8038-98a1426b41ac`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:11,894::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:11,894::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:11,894::task::993::Storage.TaskManager.Task::(_decref) Task=`a12b7afb-03ab-4ebe-8038-98a1426b41ac`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:13,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:13,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016634 s, 61.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:15,401::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:37:15,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`5627dd16-37c2-46a0-bfc0-a0207ba7f371`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:15,402::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:37:15,402::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:15,402::task::1191::Storage.TaskManager.Task::(prepare) Task=`5627dd16-37c2-46a0-bfc0-a0207ba7f371`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:15,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`5627dd16-37c2-46a0-bfc0-a0207ba7f371`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:15,402::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:15,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:15,403::task::993::Storage.TaskManager.Task::(_decref) Task=`5627dd16-37c2-46a0-bfc0-a0207ba7f371`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:37:15,434::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:37:15,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6f67295-8c11-4a3f-a8d3-e7d5ee5673c9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:37:15,434::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:37:15,434::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1c3d0360-e0f6-461f-91c4-e1f733f4ea21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:37:15,434::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:37:15,435::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:37:15,435::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1c3d0360-e0f6-461f-91c4-e1f733f4ea21`::Granted request >Thread-46::DEBUG::2014-08-22 09:37:15,435::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d6f67295-8c11-4a3f-a8d3-e7d5ee5673c9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:37:15,435::task::993::Storage.TaskManager.Task::(_decref) Task=`d6f67295-8c11-4a3f-a8d3-e7d5ee5673c9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:37:15,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:15,437::task::1191::Storage.TaskManager.Task::(prepare) Task=`d6f67295-8c11-4a3f-a8d3-e7d5ee5673c9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:15,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6f67295-8c11-4a3f-a8d3-e7d5ee5673c9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:37:15,437::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:37:15,437::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:37:15,437::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:37:15,437::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:37:15,438::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:37:15,438::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:37:15,438::task::993::Storage.TaskManager.Task::(_decref) Task=`d6f67295-8c11-4a3f-a8d3-e7d5ee5673c9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:15,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:15,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117227 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:17,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:17,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116118 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) (932) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:17,641::__init__::225::IOProcess::(_processLogs) (932) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:17,642::__init__::225::IOProcess::(_processLogs) (932) Queuing response >Thread-21::DEBUG::2014-08-22 09:37:17,643::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:17,643::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:17,649::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:37:17,650::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187694 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:17,650::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:17,650::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:17,651::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:17,651::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:17,651::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:17,651::__init__::225::IOProcess::(_processLogs) (933) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:17,651::__init__::225::IOProcess::(_processLogs) (933) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:17,652::__init__::225::IOProcess::(_processLogs) (933) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:17,652::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:17,653::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:18,327::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:18,327::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:18,327::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:18,327::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:18,327::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:18,328::__init__::225::IOProcess::(_processLogs) (934) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:18,328::__init__::225::IOProcess::(_processLogs) (934) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:18,328::__init__::225::IOProcess::(_processLogs) (934) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:18,328::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:37:18,329::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:37:18,335::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000267609 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:18,335::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:18,336::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:18,336::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:18,336::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:18,336::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:18,336::__init__::225::IOProcess::(_processLogs) (935) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:18,336::__init__::225::IOProcess::(_processLogs) (935) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:18,337::__init__::225::IOProcess::(_processLogs) (935) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:18,337::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:18,337::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) (936) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) (936) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) (936) Building response >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) (936) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:18,338::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:18,339::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:18,339::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:37:18,339::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:18,339::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) (937) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) (937) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) (937) Building response >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) (937) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:18,340::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:18,341::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:37:18,341::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:18,341::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:18,341::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) (938) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) (938) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) (938) Building response >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) (938) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:18,342::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:37:19,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:19,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116633 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:21,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:21,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117103 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:23,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:23,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115994 s, 88.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:25,489::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:37:25,489::task::595::Storage.TaskManager.Task::(_updateState) Task=`360b065a-7199-4da5-92b2-50b5a8fd874f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:25,490::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:37:25,490::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:25,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`360b065a-7199-4da5-92b2-50b5a8fd874f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:25,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`360b065a-7199-4da5-92b2-50b5a8fd874f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:25,490::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:25,490::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:25,490::task::993::Storage.TaskManager.Task::(_decref) Task=`360b065a-7199-4da5-92b2-50b5a8fd874f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:37:25,544::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:37:25,544::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ad7d394-bf3c-4751-9377-fb05bd931d04`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:37:25,545::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:37:25,545::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`76165423-5dea-4a1f-a86c-5295588116be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:37:25,545::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:37:25,545::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:37:25,545::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`76165423-5dea-4a1f-a86c-5295588116be`::Granted request >Thread-46::DEBUG::2014-08-22 09:37:25,545::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7ad7d394-bf3c-4751-9377-fb05bd931d04`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:37:25,545::task::993::Storage.TaskManager.Task::(_decref) Task=`7ad7d394-bf3c-4751-9377-fb05bd931d04`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:37:25,547::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:25,548::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ad7d394-bf3c-4751-9377-fb05bd931d04`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:25,548::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ad7d394-bf3c-4751-9377-fb05bd931d04`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:37:25,548::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:37:25,548::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:37:25,548::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:37:25,548::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:37:25,548::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:37:25,548::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:37:25,548::task::993::Storage.TaskManager.Task::(_decref) Task=`7ad7d394-bf3c-4751-9377-fb05bd931d04`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:25,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:25,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119239 s, 85.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:27,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`4427b16e-01d6-4feb-ad3e-681371eb17ea`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:27,108::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:37:27,108::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187694', 'lastCheck': '9.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000267609', 'lastCheck': '8.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:27,108::task::1191::Storage.TaskManager.Task::(prepare) Task=`4427b16e-01d6-4feb-ad3e-681371eb17ea`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187694', 'lastCheck': '9.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000267609', 'lastCheck': '8.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:27,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`4427b16e-01d6-4feb-ad3e-681371eb17ea`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:27,108::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:27,108::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:27,108::task::993::Storage.TaskManager.Task::(_decref) Task=`4427b16e-01d6-4feb-ad3e-681371eb17ea`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) (939) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:27,653::__init__::225::IOProcess::(_processLogs) (939) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:27,654::__init__::225::IOProcess::(_processLogs) (939) Queuing response >Thread-21::DEBUG::2014-08-22 09:37:27,655::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:27,655::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:27,661::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:37:27,662::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187395 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:27,662::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:27,662::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:27,663::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:27,663::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:27,663::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:27,663::__init__::225::IOProcess::(_processLogs) (940) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:27,663::__init__::225::IOProcess::(_processLogs) (940) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:27,664::__init__::225::IOProcess::(_processLogs) (940) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:27,664::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:27,664::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:37:27,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:27,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115659 s, 88.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:28,342::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:28,342::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:28,342::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:28,342::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:28,343::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:28,343::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:28,343::__init__::225::IOProcess::(_processLogs) (941) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:28,343::__init__::225::IOProcess::(_processLogs) (941) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:28,343::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(941) Building response >Thread-17::DEBUG::2014-08-22 09:37:28,344::__init__::225::IOProcess::(_processLogs) (941) Queuing response >Thread-22::DEBUG::2014-08-22 09:37:28,344::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:28,344::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:28,350::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:37:28,351::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000263465 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:28,352::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:28,352::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:28,352::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:28,352::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:28,352::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:28,352::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:28,353::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:28,353::__init__::225::IOProcess::(_processLogs) (942) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:28,353::__init__::225::IOProcess::(_processLogs) (942) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:28,353::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(942) Building response >Thread-17::DEBUG::2014-08-22 09:37:28,354::__init__::225::IOProcess::(_processLogs) (942) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:28,354::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:28,354::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:28,354::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:28,354::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:37:28,355::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:28,355::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:28,355::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:28,355::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:28,355::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:28,355::__init__::225::IOProcess::(_processLogs) (943) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) (943) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) (943) Building response >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) (943) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:37:28,356::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) (944) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) (944) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) (944) Building response >Thread-17::DEBUG::2014-08-22 09:37:28,357::__init__::225::IOProcess::(_processLogs) (944) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:28,358::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:28,358::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:28,358::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:28,358::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:37:28,358::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:28,358::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) (945) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) (945) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) (945) Building response >Thread-17::DEBUG::2014-08-22 09:37:28,359::__init__::225::IOProcess::(_processLogs) (945) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:28,360::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:28,360::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:37:29,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:29,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116872 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:31,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:31,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011588 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:33,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:33,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120646 s, 84.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:35,600::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:37:35,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ef6a3a2-119a-4450-9f48-5af4eff756dc`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:35,600::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:37:35,601::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:35,601::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ef6a3a2-119a-4450-9f48-5af4eff756dc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:35,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ef6a3a2-119a-4450-9f48-5af4eff756dc`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:35,601::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:35,601::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:35,601::task::993::Storage.TaskManager.Task::(_decref) Task=`4ef6a3a2-119a-4450-9f48-5af4eff756dc`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:37:35,632::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:37:35,632::task::595::Storage.TaskManager.Task::(_updateState) Task=`94b0980d-d981-4946-b4e1-c5e7bc9218ec`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:37:35,633::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:37:35,633::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d8580b41-c559-45cb-b7db-280be93b470e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:37:35,633::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:37:35,633::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:37:35,633::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d8580b41-c559-45cb-b7db-280be93b470e`::Granted request >Thread-46::DEBUG::2014-08-22 09:37:35,633::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`94b0980d-d981-4946-b4e1-c5e7bc9218ec`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:37:35,634::task::993::Storage.TaskManager.Task::(_decref) Task=`94b0980d-d981-4946-b4e1-c5e7bc9218ec`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:37:35,635::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:35,636::task::1191::Storage.TaskManager.Task::(prepare) Task=`94b0980d-d981-4946-b4e1-c5e7bc9218ec`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:35,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`94b0980d-d981-4946-b4e1-c5e7bc9218ec`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:37:35,636::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:37:35,636::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:37:35,636::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:37:35,636::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:37:35,636::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:37:35,636::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:37:35,636::task::993::Storage.TaskManager.Task::(_decref) Task=`94b0980d-d981-4946-b4e1-c5e7bc9218ec`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:35,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:36,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116428 s, 88.0 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:37:37,664::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:37:37,666::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,666::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:37:37,666::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,666::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,667::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,667::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,667::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,667::__init__::225::IOProcess::(_processLogs) (946) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:37:37,667::__init__::225::IOProcess::(_processLogs) (946) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:37:37,667::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(946) Building response >Thread-17::DEBUG::2014-08-22 09:37:37,668::__init__::225::IOProcess::(_processLogs) (946) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,669::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:37,669::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:37:37,670::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,671::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:37:37,671::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,672::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,673::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,673::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,674::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,675::__init__::225::IOProcess::(_processLogs) (947) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:37:37,675::__init__::225::IOProcess::(_processLogs) (947) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 09:37:37,676::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:37:37,677::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(947) Building response >Thread-17::DEBUG::2014-08-22 09:37:37,677::__init__::225::IOProcess::(_processLogs) (947) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,677::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Message size is 169 >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,678::__init__::225::IOProcess::(_processLogs) (948) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:37:37,679::__init__::225::IOProcess::(_processLogs) (948) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:37:37,706::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(948) Building response >Thread-17::DEBUG::2014-08-22 09:37:37,706::__init__::225::IOProcess::(_processLogs) (948) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,706::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:37:37,706::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:37:37,707::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:37,707::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,707::__init__::225::IOProcess::(_processLogs) Message size is 193 >Thread-17::DEBUG::2014-08-22 09:37:37,707::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,707::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,708::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,708::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,708::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,708::__init__::225::IOProcess::(_processLogs) (949) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:37:37,708::__init__::225::IOProcess::(_processLogs) (949) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:37:37,708::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(949) Building response >Thread-21::DEBUG::2014-08-22 09:37:37,708::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:37:37,709::__init__::225::IOProcess::(_processLogs) (949) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,709::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:37,710::__init__::225::IOProcess::(_processLogs) Sending response sized 603 >Thread-17::DEBUG::2014-08-22 09:37:37,710::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,710::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:37:37,710::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,710::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) (950) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) (950) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) (950) Building response >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) (950) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:37,711::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) (951) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) (951) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:37:37,712::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(951) Building response >Thread-21::DEBUG::2014-08-22 09:37:37,712::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:37:37,713::__init__::225::IOProcess::(_processLogs) (951) Queuing response >Thread-21::INFO::2014-08-22 09:37:37,713::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:37:37,713::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 09:37:37,713::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:37:37,714::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-17::DEBUG::2014-08-22 09:37:37,714::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,714::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:37,715::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,716::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:37:37,716::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:37,716::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) (952) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) (952) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) (952) Building response >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) (952) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:37,722::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:37:37,723::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.0015992 s, 248 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) (953) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:37,724::__init__::225::IOProcess::(_processLogs) (953) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:37,725::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(953) Building response >Thread-17::DEBUG::2014-08-22 09:37:37,726::__init__::225::IOProcess::(_processLogs) (953) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:37,726::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:37,726::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:37:38,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:38,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117719 s, 87.0 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-08-22 09:37:38,359::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:37:38,361::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,361::__init__::225::IOProcess::(_processLogs) Message size is 140 >Thread-17::DEBUG::2014-08-22 09:37:38,361::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,361::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,361::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,362::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,362::__init__::225::IOProcess::(_processLogs) (954) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:37:38,362::__init__::225::IOProcess::(_processLogs) (954) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:37:38,362::__init__::225::IOProcess::(_processLogs) (954) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,362::__init__::225::IOProcess::(_processLogs) (954) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,362::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Sending response sized 190 >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Message size is 142 >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,363::__init__::225::IOProcess::(_processLogs) (955) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:37:38,364::__init__::225::IOProcess::(_processLogs) (955) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:37:38,364::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(955) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,364::__init__::225::IOProcess::(_processLogs) (955) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,365::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,366::__init__::225::IOProcess::(_processLogs) Sending response sized 192 >Thread-22::DEBUG::2014-08-22 09:37:38,367::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Message size is 171 >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,368::__init__::225::IOProcess::(_processLogs) (956) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:37:38,369::__init__::225::IOProcess::(_processLogs) (956) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:37:38,398::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(956) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,398::__init__::225::IOProcess::(_processLogs) (956) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,398::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:37:38,398::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:37:38,399::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:38,399::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,399::__init__::225::IOProcess::(_processLogs) Message size is 195 >Thread-17::DEBUG::2014-08-22 09:37:38,399::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,399::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,400::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,400::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,400::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,400::__init__::225::IOProcess::(_processLogs) (957) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:37:38,400::__init__::225::IOProcess::(_processLogs) (957) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:37:38,401::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(957) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,401::__init__::225::IOProcess::(_processLogs) (957) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,402::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:37:38,402::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:37:38,402::__init__::225::IOProcess::(_processLogs) Sending response sized 831 >Thread-17::DEBUG::2014-08-22 09:37:38,403::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,403::__init__::225::IOProcess::(_processLogs) Message size is 188 >Thread-17::DEBUG::2014-08-22 09:37:38,403::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,403::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) (958) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) (958) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) (958) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) (958) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:38,404::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) Message size is 182 >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) (959) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) (959) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) (959) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,405::__init__::225::IOProcess::(_processLogs) (959) Queuing response >Thread-22::DEBUG::2014-08-22 09:37:38,406::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:37:38,406::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::INFO::2014-08-22 09:37:38,406::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:37:38,406::__init__::225::IOProcess::(_processLogs) Sending response sized 75 >Thread-22::INFO::2014-08-22 09:37:38,407::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:37:38,407::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,407::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:38,407::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,408::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:37:38,408::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:38,408::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,414::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:37:38,415::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000289921 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:38,415::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,416::__init__::225::IOProcess::(_processLogs) (960) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:38,416::__init__::225::IOProcess::(_processLogs) (960) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:38,416::__init__::225::IOProcess::(_processLogs) (960) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,416::__init__::225::IOProcess::(_processLogs) (960) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,417::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,418::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,418::__init__::225::IOProcess::(_processLogs) (961) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:38,418::__init__::225::IOProcess::(_processLogs) (961) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:38,418::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(961) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,419::__init__::225::IOProcess::(_processLogs) (961) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,419::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,419::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:38,419::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,419::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:37:38,420::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,420::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,420::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,420::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,420::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,420::__init__::225::IOProcess::(_processLogs) (962) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) (962) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) (962) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) (962) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:37:38,421::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) (963) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) (963) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) (963) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,422::__init__::225::IOProcess::(_processLogs) (963) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,423::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,423::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:38,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:38,423::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:37:38,423::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:38,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:38,424::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:38,424::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:38,424::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:38,424::__init__::225::IOProcess::(_processLogs) (964) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:38,424::__init__::225::IOProcess::(_processLogs) (964) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:38,424::__init__::225::IOProcess::(_processLogs) (964) Building response >Thread-17::DEBUG::2014-08-22 09:37:38,425::__init__::225::IOProcess::(_processLogs) (964) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:38,425::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:38,425::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:37:40,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:40,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011814 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:42,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:42,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116538 s, 87.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:42,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dd63591-92ce-45c1-b45d-8702aef8a8a3`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:42,318::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:37:42,318::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0015992', 'lastCheck': '4.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000289921', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:42,318::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dd63591-92ce-45c1-b45d-8702aef8a8a3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.0015992', 'lastCheck': '4.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000289921', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:42,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dd63591-92ce-45c1-b45d-8702aef8a8a3`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:42,318::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:42,318::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:42,318::task::993::Storage.TaskManager.Task::(_decref) Task=`9dd63591-92ce-45c1-b45d-8702aef8a8a3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:44,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:44,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115498 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:45,684::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:37:45,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7470608-fece-415b-ac3b-94128551c60e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:45,685::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:37:45,685::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:45,685::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7470608-fece-415b-ac3b-94128551c60e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:45,685::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7470608-fece-415b-ac3b-94128551c60e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:45,685::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:45,685::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:45,685::task::993::Storage.TaskManager.Task::(_decref) Task=`b7470608-fece-415b-ac3b-94128551c60e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:37:45,711::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:37:45,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef49e175-3d3d-4119-8724-6c7a57f4e611`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:37:45,711::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:37:45,711::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2fcddb15-b5fa-4b24-9f38-07ea29a83071`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:37:45,711::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:37:45,712::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:37:45,712::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2fcddb15-b5fa-4b24-9f38-07ea29a83071`::Granted request >Thread-46::DEBUG::2014-08-22 09:37:45,712::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ef49e175-3d3d-4119-8724-6c7a57f4e611`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:37:45,712::task::993::Storage.TaskManager.Task::(_decref) Task=`ef49e175-3d3d-4119-8724-6c7a57f4e611`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:37:45,714::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:45,714::task::1191::Storage.TaskManager.Task::(prepare) Task=`ef49e175-3d3d-4119-8724-6c7a57f4e611`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:45,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef49e175-3d3d-4119-8724-6c7a57f4e611`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:37:45,714::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:37:45,714::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:37:45,714::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:37:45,714::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:37:45,715::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:37:45,715::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:37:45,715::task::993::Storage.TaskManager.Task::(_decref) Task=`ef49e175-3d3d-4119-8724-6c7a57f4e611`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:46,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:46,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119202 s, 85.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:47,726::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:47,726::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) (965) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:47,727::__init__::225::IOProcess::(_processLogs) (965) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:47,728::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(965) Building response >Thread-21::DEBUG::2014-08-22 09:37:47,728::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:47,728::__init__::225::IOProcess::(_processLogs) (965) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:47,735::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:37:47,735::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00018616 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:47,736::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:47,737::__init__::225::IOProcess::(_processLogs) (966) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:47,738::__init__::225::IOProcess::(_processLogs) (966) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:47,738::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(966) Building response >Thread-17::DEBUG::2014-08-22 09:37:47,739::__init__::225::IOProcess::(_processLogs) (966) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:47,739::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:47,739::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Dummy-45::DEBUG::2014-08-22 09:37:48,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:48,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117543 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) (967) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:48,425::__init__::225::IOProcess::(_processLogs) (967) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:48,426::__init__::225::IOProcess::(_processLogs) (967) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:48,426::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:37:48,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:48,427::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:37:48,433::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214733 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:48,433::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:48,434::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:48,434::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:48,434::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:48,434::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:48,434::__init__::225::IOProcess::(_processLogs) (968) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:48,434::__init__::225::IOProcess::(_processLogs) (968) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:48,435::__init__::225::IOProcess::(_processLogs) (968) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:48,435::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:48,435::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:48,436::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:37:48,436::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:48,436::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:48,436::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:48,436::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) (969) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) (969) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) (969) Building response >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) (969) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:37:48,437::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) (970) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) (970) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) (970) Building response >Thread-17::DEBUG::2014-08-22 09:37:48,438::__init__::225::IOProcess::(_processLogs) (970) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:48,439::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:48,439::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:48,439::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:37:48,439::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:48,439::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) (971) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) (971) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) (971) Building response >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) (971) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:48,440::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:37:50,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:50,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011861 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:52,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:52,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116732 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:37:54,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:54,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118258 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:55,764::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:37:55,764::task::595::Storage.TaskManager.Task::(_updateState) Task=`aebdf061-47e6-4ab5-a077-6715193c4c83`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:55,764::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:37:55,765::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:55,765::task::1191::Storage.TaskManager.Task::(prepare) Task=`aebdf061-47e6-4ab5-a077-6715193c4c83`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:37:55,765::task::595::Storage.TaskManager.Task::(_updateState) Task=`aebdf061-47e6-4ab5-a077-6715193c4c83`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:55,765::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:55,765::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:55,765::task::993::Storage.TaskManager.Task::(_decref) Task=`aebdf061-47e6-4ab5-a077-6715193c4c83`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:37:55,800::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:37:55,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb619a67-9f36-4eb3-91c6-382499181d5e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:37:55,800::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:37:55,800::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6bcf841f-19f9-4a04-bd31-1ffc2a1450e0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:37:55,800::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:37:55,800::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:37:55,800::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6bcf841f-19f9-4a04-bd31-1ffc2a1450e0`::Granted request >Thread-46::DEBUG::2014-08-22 09:37:55,801::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eb619a67-9f36-4eb3-91c6-382499181d5e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:37:55,801::task::993::Storage.TaskManager.Task::(_decref) Task=`eb619a67-9f36-4eb3-91c6-382499181d5e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:37:55,803::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:55,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb619a67-9f36-4eb3-91c6-382499181d5e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:37:55,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb619a67-9f36-4eb3-91c6-382499181d5e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:37:55,803::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:37:55,803::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:37:55,803::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:37:55,803::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:37:55,803::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:37:55,803::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:37:55,804::task::993::Storage.TaskManager.Task::(_decref) Task=`eb619a67-9f36-4eb3-91c6-382499181d5e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:37:56,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:56,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112734 s, 90.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:37:57,508::task::595::Storage.TaskManager.Task::(_updateState) Task=`3390031a-2dc4-4be5-8127-347beec06cd5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:37:57,508::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:37:57,508::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00018616', 'lastCheck': '9.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214733', 'lastCheck': '9.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:57,508::task::1191::Storage.TaskManager.Task::(prepare) Task=`3390031a-2dc4-4be5-8127-347beec06cd5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00018616', 'lastCheck': '9.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214733', 'lastCheck': '9.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:37:57,508::task::595::Storage.TaskManager.Task::(_updateState) Task=`3390031a-2dc4-4be5-8127-347beec06cd5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:37:57,509::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:37:57,509::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:37:57,509::task::993::Storage.TaskManager.Task::(_decref) Task=`3390031a-2dc4-4be5-8127-347beec06cd5`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:37:57,739::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) (972) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:57,740::__init__::225::IOProcess::(_processLogs) (972) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:57,741::__init__::225::IOProcess::(_processLogs) (972) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:57,741::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:37:57,741::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:57,742::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:37:57,748::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000188827 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) (973) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:57,749::__init__::225::IOProcess::(_processLogs) (973) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:57,750::__init__::225::IOProcess::(_processLogs) (973) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:57,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:57,751::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:58,440::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:58,440::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:58,441::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:58,441::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:58,441::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:58,441::__init__::225::IOProcess::(_processLogs) (974) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:58,441::__init__::225::IOProcess::(_processLogs) (974) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:58,441::__init__::225::IOProcess::(_processLogs) (974) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:58,442::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:37:58,442::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:37:58,442::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:37:58,448::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000158804 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) (975) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:37:58,449::__init__::225::IOProcess::(_processLogs) (975) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:37:58,450::__init__::225::IOProcess::(_processLogs) (975) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:58,451::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:58,451::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:37:58,451::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:37:58,451::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) (976) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) (976) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) (976) Building response >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) (976) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:58,452::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) (977) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:58,453::__init__::225::IOProcess::(_processLogs) (977) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) (977) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:37:58,454::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:37:58,455::__init__::225::IOProcess::(_processLogs) (978) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:37:58,455::__init__::225::IOProcess::(_processLogs) (978) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:37:58,455::__init__::225::IOProcess::(_processLogs) (978) Queuing response >Thread-17::DEBUG::2014-08-22 09:37:58,455::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:37:58,456::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:37:58,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:37:58,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117266 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:00,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:00,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01176 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:02,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:02,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211135 s, 48.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:04,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:04,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118785 s, 86.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:05,901::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:38:05,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`f98be3e8-8efc-4b9d-ae3b-8284d41863cb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:05,902::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:38:05,902::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:05,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`f98be3e8-8efc-4b9d-ae3b-8284d41863cb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:05,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`f98be3e8-8efc-4b9d-ae3b-8284d41863cb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:05,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:05,902::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:05,903::task::993::Storage.TaskManager.Task::(_decref) Task=`f98be3e8-8efc-4b9d-ae3b-8284d41863cb`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:38:05,948::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:38:05,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`b02ec5cd-025e-483b-b34b-27e3aa82ff00`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:38:05,949::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:38:05,949::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0e532774-d61b-4a8e-bd70-044053aef292`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:38:05,949::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:38:05,949::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:38:05,949::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0e532774-d61b-4a8e-bd70-044053aef292`::Granted request >Thread-46::DEBUG::2014-08-22 09:38:05,949::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b02ec5cd-025e-483b-b34b-27e3aa82ff00`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:38:05,950::task::993::Storage.TaskManager.Task::(_decref) Task=`b02ec5cd-025e-483b-b34b-27e3aa82ff00`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:38:05,951::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:05,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`b02ec5cd-025e-483b-b34b-27e3aa82ff00`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:05,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`b02ec5cd-025e-483b-b34b-27e3aa82ff00`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:38:05,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:38:05,952::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:38:05,952::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:38:05,952::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:38:05,952::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:38:05,952::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:38:05,952::task::993::Storage.TaskManager.Task::(_decref) Task=`b02ec5cd-025e-483b-b34b-27e3aa82ff00`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:38:06,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:06,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114883 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:07,751::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:38:07,751::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:07,751::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:07,751::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:07,751::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:07,752::__init__::225::IOProcess::(_processLogs) (979) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:07,752::__init__::225::IOProcess::(_processLogs) (979) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:07,752::__init__::225::IOProcess::(_processLogs) (979) Queuing response >Thread-21::DEBUG::2014-08-22 09:38:07,753::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:07,753::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:07,759::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:38:07,760::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000173658 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) (980) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:07,761::__init__::225::IOProcess::(_processLogs) (980) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:07,762::__init__::225::IOProcess::(_processLogs) (980) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:07,763::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:07,763::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:38:08,456::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:38:08,456::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:08,456::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:08,456::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:08,456::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:08,457::__init__::225::IOProcess::(_processLogs) (981) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:08,457::__init__::225::IOProcess::(_processLogs) (981) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:08,457::__init__::225::IOProcess::(_processLogs) (981) Building response >Thread-17::DEBUG::2014-08-22 09:38:08,457::__init__::225::IOProcess::(_processLogs) (981) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:08,457::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:38:08,457::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:38:08,463::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000264583 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) (982) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:08,464::__init__::225::IOProcess::(_processLogs) (982) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:08,465::__init__::225::IOProcess::(_processLogs) (982) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:08,466::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:08,466::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:38:08,466::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:38:08,466::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:08,466::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) (983) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) (983) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) (983) Building response >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) (983) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:38:08,467::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) (984) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) (984) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:08,468::__init__::225::IOProcess::(_processLogs) (984) Building response >Thread-17::DEBUG::2014-08-22 09:38:08,469::__init__::225::IOProcess::(_processLogs) (984) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:08,469::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:08,469::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:38:08,469::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:38:08,469::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:08,469::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) (985) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) (985) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) (985) Building response >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) (985) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:08,470::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:08,471::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:38:08,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:08,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011868 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:10,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:11,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115824 s, 88.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:12,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`8be40252-2ba0-4178-a51c-9a37ec23dee8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:12,720::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:38:12,720::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000173658', 'lastCheck': '5.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264583', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:12,720::task::1191::Storage.TaskManager.Task::(prepare) Task=`8be40252-2ba0-4178-a51c-9a37ec23dee8`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000173658', 'lastCheck': '5.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264583', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:12,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`8be40252-2ba0-4178-a51c-9a37ec23dee8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:12,720::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:12,720::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:12,720::task::993::Storage.TaskManager.Task::(_decref) Task=`8be40252-2ba0-4178-a51c-9a37ec23dee8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:38:13,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:13,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116182 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:15,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:15,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116689 s, 87.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:15,994::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:38:15,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`180eb2b8-ca79-4a46-a27b-ac88469aa38f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:15,995::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:38:15,995::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:15,995::task::1191::Storage.TaskManager.Task::(prepare) Task=`180eb2b8-ca79-4a46-a27b-ac88469aa38f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:15,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`180eb2b8-ca79-4a46-a27b-ac88469aa38f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:15,995::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:15,995::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:15,995::task::993::Storage.TaskManager.Task::(_decref) Task=`180eb2b8-ca79-4a46-a27b-ac88469aa38f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:38:16,105::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:38:16,105::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee3c363f-9369-43b5-88aa-ea3ff6428d1f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:38:16,106::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:38:16,106::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bd8509ff-44c3-4d98-9565-304dc0f3d686`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:38:16,106::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:38:16,106::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:38:16,106::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bd8509ff-44c3-4d98-9565-304dc0f3d686`::Granted request >Thread-46::DEBUG::2014-08-22 09:38:16,107::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ee3c363f-9369-43b5-88aa-ea3ff6428d1f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:38:16,107::task::993::Storage.TaskManager.Task::(_decref) Task=`ee3c363f-9369-43b5-88aa-ea3ff6428d1f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:38:16,109::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:16,109::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee3c363f-9369-43b5-88aa-ea3ff6428d1f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:16,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee3c363f-9369-43b5-88aa-ea3ff6428d1f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:38:16,109::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:38:16,109::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:38:16,109::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:38:16,109::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:38:16,109::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:38:16,109::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:38:16,109::task::993::Storage.TaskManager.Task::(_decref) Task=`ee3c363f-9369-43b5-88aa-ea3ff6428d1f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:38:17,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:17,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118013 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:17,763::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:38:17,763::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:17,763::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:17,763::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:17,763::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:17,764::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:17,764::__init__::225::IOProcess::(_processLogs) (986) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:17,764::__init__::225::IOProcess::(_processLogs) (986) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:17,764::__init__::225::IOProcess::(_processLogs) DEBUG|(986) Building response >Thread-21::DEBUG::2014-08-22 09:38:17,765::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:17,765::__init__::225::IOProcess::(_processLogs) (986) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:17,771::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:38:17,772::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000229941 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:17,772::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) (987) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) (987) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) (987) Building response >Thread-17::DEBUG::2014-08-22 09:38:17,774::__init__::225::IOProcess::(_processLogs) (987) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:17,775::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:38:17,776::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:38:18,470::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) (988) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) (988) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:18,471::__init__::225::IOProcess::(_processLogs) DEBUG|(988) Building response >Thread-17::DEBUG::2014-08-22 09:38:18,472::__init__::225::IOProcess::(_processLogs) (988) Queuing response >Thread-22::DEBUG::2014-08-22 09:38:18,472::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:18,473::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:18,479::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:38:18,479::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000263572 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:18,480::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:18,480::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:38:18,480::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) (989) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) (989) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) (989) Building response >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) (989) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:18,481::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:18,482::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:38:18,483::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:18,483::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:38:18,483::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:18,483::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:18,483::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) (990) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) (990) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) (990) Building response >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) (990) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:18,484::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:38:18,485::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:18,485::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:18,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:18,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:18,485::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:18,485::__init__::225::IOProcess::(_processLogs) (991) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) (991) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) (991) Building response >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) (991) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:38:18,486::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:18,487::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:18,487::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:18,487::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:18,487::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:18,487::__init__::225::IOProcess::(_processLogs) (992) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:18,488::__init__::225::IOProcess::(_processLogs) (992) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:18,488::__init__::225::IOProcess::(_processLogs) (992) Building response >Thread-17::DEBUG::2014-08-22 09:38:18,488::__init__::225::IOProcess::(_processLogs) (992) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:18,488::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:18,488::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:38:19,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:19,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011842 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:21,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:21,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115005 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:23,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:23,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114891 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:25,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:25,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147168 s, 69.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:26,161::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:38:26,161::task::595::Storage.TaskManager.Task::(_updateState) Task=`985c2508-b67e-42aa-a3ca-9648126992ae`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:26,161::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:38:26,162::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:26,162::task::1191::Storage.TaskManager.Task::(prepare) Task=`985c2508-b67e-42aa-a3ca-9648126992ae`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:26,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`985c2508-b67e-42aa-a3ca-9648126992ae`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:26,162::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:26,162::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:26,162::task::993::Storage.TaskManager.Task::(_decref) Task=`985c2508-b67e-42aa-a3ca-9648126992ae`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:38:26,200::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:38:26,200::task::595::Storage.TaskManager.Task::(_updateState) Task=`166a9039-5269-4102-b847-6810349c7951`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:38:26,200::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:38:26,200::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`512d95b4-70f7-4d27-927d-c91e05c29679`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:38:26,200::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:38:26,201::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:38:26,201::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`512d95b4-70f7-4d27-927d-c91e05c29679`::Granted request >Thread-46::DEBUG::2014-08-22 09:38:26,201::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`166a9039-5269-4102-b847-6810349c7951`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:38:26,201::task::993::Storage.TaskManager.Task::(_decref) Task=`166a9039-5269-4102-b847-6810349c7951`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:38:26,203::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:26,203::task::1191::Storage.TaskManager.Task::(prepare) Task=`166a9039-5269-4102-b847-6810349c7951`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:26,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`166a9039-5269-4102-b847-6810349c7951`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:38:26,203::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:38:26,203::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:38:26,203::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:38:26,204::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:38:26,204::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:38:26,204::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:38:26,204::task::993::Storage.TaskManager.Task::(_decref) Task=`166a9039-5269-4102-b847-6810349c7951`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:38:27,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:27,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011423 s, 89.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:27,776::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:27,776::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) (993) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:27,777::__init__::225::IOProcess::(_processLogs) (993) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:27,778::__init__::225::IOProcess::(_processLogs) (993) Queuing response >Thread-21::DEBUG::2014-08-22 09:38:27,778::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:27,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:27,785::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-21::DEBUG::2014-08-22 09:38:27,786::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000145411 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:27,786::__init__::225::IOProcess::(_processLogs) Message size is 160 >Thread-17::DEBUG::2014-08-22 09:38:27,786::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:27,786::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:27,786::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:27,787::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:27,787::__init__::225::IOProcess::(_processLogs) (994) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:27,787::__init__::225::IOProcess::(_processLogs) (994) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:27,787::__init__::225::IOProcess::(_processLogs) (994) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:27,788::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:27,788::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-47::DEBUG::2014-08-22 09:38:27,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`3119af8f-41e0-4ead-8044-95dec9774580`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:27,898::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:38:27,898::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000145411', 'lastCheck': '0.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000263572', 'lastCheck': '9.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:27,898::task::1191::Storage.TaskManager.Task::(prepare) Task=`3119af8f-41e0-4ead-8044-95dec9774580`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000145411', 'lastCheck': '0.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000263572', 'lastCheck': '9.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:27,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`3119af8f-41e0-4ead-8044-95dec9774580`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:27,898::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:27,898::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:27,898::task::993::Storage.TaskManager.Task::(_decref) Task=`3119af8f-41e0-4ead-8044-95dec9774580`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) (995) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:28,488::__init__::225::IOProcess::(_processLogs) (995) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:28,489::__init__::225::IOProcess::(_processLogs) (995) Building response >Thread-17::DEBUG::2014-08-22 09:38:28,489::__init__::225::IOProcess::(_processLogs) (995) Queuing response >Thread-22::DEBUG::2014-08-22 09:38:28,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:28,490::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:28,496::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-22::DEBUG::2014-08-22 09:38:28,496::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000197733 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:28,497::__init__::225::IOProcess::(_processLogs) Message size is 162 >Thread-17::DEBUG::2014-08-22 09:38:28,497::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:28,497::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:28,497::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:28,498::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:28,498::__init__::225::IOProcess::(_processLogs) (996) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:28,498::__init__::225::IOProcess::(_processLogs) (996) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:28,498::__init__::225::IOProcess::(_processLogs) (996) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:28,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:28,499::__init__::225::IOProcess::(_processLogs) Sending response sized 328 >Thread-17::DEBUG::2014-08-22 09:38:28,499::__init__::225::IOProcess::(_processLogs) Message size is 179 >Thread-17::DEBUG::2014-08-22 09:38:28,499::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) (997) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) (997) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) (997) Building response >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) (997) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:28,500::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:28,501::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:38:28,501::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:38:28,501::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:28,501::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:28,501::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) (998) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) (998) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) (998) Building response >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) (998) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) Message size is 185 >Thread-17::DEBUG::2014-08-22 09:38:28,502::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:28,503::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:28,503::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:28,503::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:28,503::__init__::225::IOProcess::(_processLogs) (999) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:28,504::__init__::225::IOProcess::(_processLogs) (999) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:28,504::__init__::225::IOProcess::(_processLogs) (999) Building response >Thread-17::DEBUG::2014-08-22 09:38:28,504::__init__::225::IOProcess::(_processLogs) (999) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:28,504::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:28,504::__init__::225::IOProcess::(_processLogs) Sending response sized 73 >Dummy-45::DEBUG::2014-08-22 09:38:29,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:29,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120446 s, 85.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:31,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:31,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117501 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:33,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:33,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116721 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:35,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:35,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118984 s, 86.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:36,260::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:38:36,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`cff0be75-b05e-4f1c-a23c-3592c63c57d8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:36,260::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:38:36,261::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:36,261::task::1191::Storage.TaskManager.Task::(prepare) Task=`cff0be75-b05e-4f1c-a23c-3592c63c57d8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:36,261::task::595::Storage.TaskManager.Task::(_updateState) Task=`cff0be75-b05e-4f1c-a23c-3592c63c57d8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:36,261::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:36,261::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:36,261::task::993::Storage.TaskManager.Task::(_decref) Task=`cff0be75-b05e-4f1c-a23c-3592c63c57d8`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:38:36,295::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:38:36,295::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca7dee8e-79dd-4ba6-8ddd-b4578a1f7e0a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:38:36,295::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:38:36,296::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e0c52a6e-2276-4153-9e3c-5b06aebd018d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:38:36,296::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:38:36,296::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:38:36,296::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e0c52a6e-2276-4153-9e3c-5b06aebd018d`::Granted request >Thread-46::DEBUG::2014-08-22 09:38:36,296::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ca7dee8e-79dd-4ba6-8ddd-b4578a1f7e0a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:38:36,296::task::993::Storage.TaskManager.Task::(_decref) Task=`ca7dee8e-79dd-4ba6-8ddd-b4578a1f7e0a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:38:36,298::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:36,298::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca7dee8e-79dd-4ba6-8ddd-b4578a1f7e0a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:36,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca7dee8e-79dd-4ba6-8ddd-b4578a1f7e0a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:38:36,299::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:38:36,299::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:38:36,299::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:38:36,299::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:38:36,299::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:38:36,299::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:38:36,299::task::993::Storage.TaskManager.Task::(_decref) Task=`ca7dee8e-79dd-4ba6-8ddd-b4578a1f7e0a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:38:37,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:37,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116992 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) (1000) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:37,789::__init__::225::IOProcess::(_processLogs) (1000) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:37,790::__init__::225::IOProcess::(_processLogs) (1000) Queuing response >Thread-21::DEBUG::2014-08-22 09:38:37,790::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:37,791::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:37,797::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:38:37,797::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000208334 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:37,798::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:38:37,798::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:37,798::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:37,798::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:37,798::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:37,798::__init__::225::IOProcess::(_processLogs) (1001) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:37,799::__init__::225::IOProcess::(_processLogs) (1001) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:37,799::__init__::225::IOProcess::(_processLogs) (1001) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:37,800::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:37,800::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) (1002) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:38,504::__init__::225::IOProcess::(_processLogs) (1002) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:38,505::__init__::225::IOProcess::(_processLogs) (1002) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:38,505::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:38:38,505::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:38,506::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:38:38,512::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000213795 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:38,512::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:38:38,512::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:38,512::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:38,513::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:38,513::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:38,513::__init__::225::IOProcess::(_processLogs) (1003) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:38,513::__init__::225::IOProcess::(_processLogs) (1003) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:38,513::__init__::225::IOProcess::(_processLogs) (1003) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:38,514::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:38,514::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:38:38,514::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:38:38,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:38,516::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:38,516::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:38,516::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:38,516::__init__::225::IOProcess::(_processLogs) (1004) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) (1004) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) (1004) Building response >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) (1004) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:38:38,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) (1005) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) (1005) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) (1005) Building response >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) (1005) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:38,518::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:38,519::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:38:38,519::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:38:38,519::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:38,519::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:38,519::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) (1006) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) (1006) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) (1006) Building response >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) (1006) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:38,520::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:38:39,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:39,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115872 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:41,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:41,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116429 s, 88.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:43,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad0a4948-e949-4855-98a8-d166eed57b15`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:43,132::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:38:43,132::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208334', 'lastCheck': '5.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000213795', 'lastCheck': '4.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:43,132::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad0a4948-e949-4855-98a8-d166eed57b15`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208334', 'lastCheck': '5.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000213795', 'lastCheck': '4.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:43,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad0a4948-e949-4855-98a8-d166eed57b15`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:43,133::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:43,133::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:43,133::task::993::Storage.TaskManager.Task::(_decref) Task=`ad0a4948-e949-4855-98a8-d166eed57b15`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:38:43,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:43,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011638 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:45,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:45,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116167 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:46,347::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:38:46,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`708cb668-a20c-4d8e-a0ec-758c8c4c104f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:46,348::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:38:46,348::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:46,348::task::1191::Storage.TaskManager.Task::(prepare) Task=`708cb668-a20c-4d8e-a0ec-758c8c4c104f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:46,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`708cb668-a20c-4d8e-a0ec-758c8c4c104f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:46,348::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:46,348::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:46,348::task::993::Storage.TaskManager.Task::(_decref) Task=`708cb668-a20c-4d8e-a0ec-758c8c4c104f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:38:46,382::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:38:46,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`9580a36b-76ab-45e9-ae6e-be6117561080`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:38:46,382::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:38:46,383::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4625064f-ee4a-4205-8454-a8e7a5b4810a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:38:46,383::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:38:46,383::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:38:46,383::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4625064f-ee4a-4205-8454-a8e7a5b4810a`::Granted request >Thread-46::DEBUG::2014-08-22 09:38:46,383::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9580a36b-76ab-45e9-ae6e-be6117561080`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:38:46,383::task::993::Storage.TaskManager.Task::(_decref) Task=`9580a36b-76ab-45e9-ae6e-be6117561080`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:38:46,385::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:46,385::task::1191::Storage.TaskManager.Task::(prepare) Task=`9580a36b-76ab-45e9-ae6e-be6117561080`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:46,386::task::595::Storage.TaskManager.Task::(_updateState) Task=`9580a36b-76ab-45e9-ae6e-be6117561080`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:38:46,386::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:38:46,386::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:38:46,386::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:38:46,386::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:38:46,386::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:38:46,386::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:38:46,386::task::993::Storage.TaskManager.Task::(_decref) Task=`9580a36b-76ab-45e9-ae6e-be6117561080`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:38:47,800::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) (1007) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:47,801::__init__::225::IOProcess::(_processLogs) (1007) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:47,802::__init__::225::IOProcess::(_processLogs) DEBUG|(1007) Building response >Thread-21::DEBUG::2014-08-22 09:38:47,802::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:47,802::__init__::225::IOProcess::(_processLogs) (1007) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:47,808::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:38:47,809::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000211686 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:47,809::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) (1008) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:47,811::__init__::225::IOProcess::(_processLogs) (1008) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:47,812::__init__::225::IOProcess::(_processLogs) DEBUG|(1008) Building response >Thread-17::DEBUG::2014-08-22 09:38:47,813::__init__::225::IOProcess::(_processLogs) (1008) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:47,813::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:47,813::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:38:47,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:47,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118298 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:48,520::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) (1009) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:48,521::__init__::225::IOProcess::(_processLogs) (1009) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:48,522::__init__::225::IOProcess::(_processLogs) (1009) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:48,522::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:38:48,522::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:48,523::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:38:48,529::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000241264 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:48,529::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:38:48,529::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:48,530::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:48,530::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:48,530::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:48,530::__init__::225::IOProcess::(_processLogs) (1010) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:48,530::__init__::225::IOProcess::(_processLogs) (1010) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:48,531::__init__::225::IOProcess::(_processLogs) (1010) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:48,531::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:48,531::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:38:48,531::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:38:48,532::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:48,532::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:48,532::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:48,532::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:48,532::__init__::225::IOProcess::(_processLogs) (1011) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) (1011) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) (1011) Building response >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) (1011) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:38:48,533::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) (1012) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) (1012) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) (1012) Building response >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) (1012) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:48,534::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:48,535::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:38:48,535::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:38:48,535::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:48,535::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) (1013) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) (1013) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) (1013) Building response >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) (1013) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:48,536::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:38:49,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:49,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118825 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:51,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:52,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141431 s, 72.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:54,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:54,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116613 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:38:56,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:56,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119866 s, 85.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:56,435::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:38:56,435::task::595::Storage.TaskManager.Task::(_updateState) Task=`794cec41-e62f-4aaa-8dac-d2749d47ea87`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:56,435::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:38:56,435::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:56,436::task::1191::Storage.TaskManager.Task::(prepare) Task=`794cec41-e62f-4aaa-8dac-d2749d47ea87`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:38:56,436::task::595::Storage.TaskManager.Task::(_updateState) Task=`794cec41-e62f-4aaa-8dac-d2749d47ea87`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:56,436::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:56,436::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:56,436::task::993::Storage.TaskManager.Task::(_decref) Task=`794cec41-e62f-4aaa-8dac-d2749d47ea87`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:38:56,460::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:38:56,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d969154-c5a6-436a-b13b-1389697c0879`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:38:56,460::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:38:56,461::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c3c75cff-b79b-420b-8b0c-aeec04cce967`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:38:56,461::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:38:56,461::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:38:56,461::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c3c75cff-b79b-420b-8b0c-aeec04cce967`::Granted request >Thread-46::DEBUG::2014-08-22 09:38:56,461::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7d969154-c5a6-436a-b13b-1389697c0879`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:38:56,461::task::993::Storage.TaskManager.Task::(_decref) Task=`7d969154-c5a6-436a-b13b-1389697c0879`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:38:56,463::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:56,463::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d969154-c5a6-436a-b13b-1389697c0879`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:38:56,464::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d969154-c5a6-436a-b13b-1389697c0879`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:38:56,464::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:38:56,464::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:38:56,464::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:38:56,464::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:38:56,464::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:38:56,464::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:38:56,464::task::993::Storage.TaskManager.Task::(_decref) Task=`7d969154-c5a6-436a-b13b-1389697c0879`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:38:57,813::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:38:57,813::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:57,813::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:57,814::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:57,814::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:57,814::__init__::225::IOProcess::(_processLogs) (1014) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:57,814::__init__::225::IOProcess::(_processLogs) (1014) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:57,814::__init__::225::IOProcess::(_processLogs) (1014) Queuing response >Thread-21::DEBUG::2014-08-22 09:38:57,815::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:57,815::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:57,822::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:38:57,822::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000180589 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:57,823::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:38:57,823::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:57,823::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:57,823::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:57,823::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:57,823::__init__::225::IOProcess::(_processLogs) (1015) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:57,824::__init__::225::IOProcess::(_processLogs) (1015) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:57,824::__init__::225::IOProcess::(_processLogs) (1015) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:57,825::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:57,825::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:38:58,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:38:58,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118381 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:38:58,472::task::595::Storage.TaskManager.Task::(_updateState) Task=`74658d3e-c9ae-4bf6-87f3-6b2f2f6ab2f7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:38:58,473::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:38:58,473::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180589', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000241264', 'lastCheck': '9.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:58,473::task::1191::Storage.TaskManager.Task::(prepare) Task=`74658d3e-c9ae-4bf6-87f3-6b2f2f6ab2f7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180589', 'lastCheck': '0.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000241264', 'lastCheck': '9.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:38:58,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`74658d3e-c9ae-4bf6-87f3-6b2f2f6ab2f7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:38:58,473::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:38:58,473::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:38:58,473::task::993::Storage.TaskManager.Task::(_decref) Task=`74658d3e-c9ae-4bf6-87f3-6b2f2f6ab2f7`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:38:58,536::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:38:58,536::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:58,536::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:58,537::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:58,537::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:58,537::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:58,537::__init__::225::IOProcess::(_processLogs) (1016) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:58,537::__init__::225::IOProcess::(_processLogs) (1016) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:58,537::__init__::225::IOProcess::(_processLogs) (1016) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:58,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:38:58,538::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:38:58,538::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:38:58,544::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000172925 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:38:58,545::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:38:58,545::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:38:58,545::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:58,545::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:58,545::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:58,546::__init__::225::IOProcess::(_processLogs) (1017) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:38:58,546::__init__::225::IOProcess::(_processLogs) (1017) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:38:58,546::__init__::225::IOProcess::(_processLogs) (1017) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:58,547::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:58,547::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:38:58,548::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:38:58,548::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:58,548::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) (1018) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) (1018) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) (1018) Building response >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) (1018) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:38:58,549::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) (1019) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) (1019) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:58,550::__init__::225::IOProcess::(_processLogs) (1019) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:38:58,551::__init__::225::IOProcess::(_processLogs) (1020) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:38:58,552::__init__::225::IOProcess::(_processLogs) (1020) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:38:58,552::__init__::225::IOProcess::(_processLogs) (1020) Queuing response >Thread-17::DEBUG::2014-08-22 09:38:58,552::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:38:58,553::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:39:00,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:00,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118346 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:02,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:02,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183679 s, 55.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:04,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:04,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116015 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:06,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:06,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114608 s, 89.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:06,522::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:39:06,522::task::595::Storage.TaskManager.Task::(_updateState) Task=`db4e3942-52ab-4b2e-a53a-5c476dfc5c89`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:06,522::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:39:06,522::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:06,522::task::1191::Storage.TaskManager.Task::(prepare) Task=`db4e3942-52ab-4b2e-a53a-5c476dfc5c89`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:06,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`db4e3942-52ab-4b2e-a53a-5c476dfc5c89`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:06,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:06,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:06,523::task::993::Storage.TaskManager.Task::(_decref) Task=`db4e3942-52ab-4b2e-a53a-5c476dfc5c89`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:39:06,613::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:39:06,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`56b6e340-9546-4c8b-9edb-7e00dcc1abfe`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:39:06,614::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:39:06,614::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b64b189f-e690-49fd-8054-c0d96784008d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:39:06,614::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:39:06,614::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:39:06,614::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b64b189f-e690-49fd-8054-c0d96784008d`::Granted request >Thread-46::DEBUG::2014-08-22 09:39:06,615::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`56b6e340-9546-4c8b-9edb-7e00dcc1abfe`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:39:06,615::task::993::Storage.TaskManager.Task::(_decref) Task=`56b6e340-9546-4c8b-9edb-7e00dcc1abfe`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:39:06,617::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:06,617::task::1191::Storage.TaskManager.Task::(prepare) Task=`56b6e340-9546-4c8b-9edb-7e00dcc1abfe`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:06,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`56b6e340-9546-4c8b-9edb-7e00dcc1abfe`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:39:06,617::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:39:06,617::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:39:06,617::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:39:06,617::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:39:06,617::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:39:06,617::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:39:06,618::task::993::Storage.TaskManager.Task::(_decref) Task=`56b6e340-9546-4c8b-9edb-7e00dcc1abfe`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:39:07,825::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:07,825::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:07,826::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:07,826::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:07,826::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:07,826::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:07,826::__init__::225::IOProcess::(_processLogs) (1021) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:07,826::__init__::225::IOProcess::(_processLogs) (1021) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:07,827::__init__::225::IOProcess::(_processLogs) (1021) Queuing response >Thread-21::DEBUG::2014-08-22 09:39:07,827::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:07,827::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:07,834::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:39:07,834::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000184045 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:07,835::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:07,835::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:07,835::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:07,835::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:07,835::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:07,836::__init__::225::IOProcess::(_processLogs) (1022) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:07,836::__init__::225::IOProcess::(_processLogs) (1022) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:07,836::__init__::225::IOProcess::(_processLogs) (1022) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:07,837::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:07,837::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:39:08,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:08,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119194 s, 85.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:08,553::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:08,553::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:08,553::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:08,553::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:08,553::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:08,554::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:08,554::__init__::225::IOProcess::(_processLogs) (1023) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:08,554::__init__::225::IOProcess::(_processLogs) (1023) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:08,554::__init__::225::IOProcess::(_processLogs) (1023) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:08,555::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:39:08,555::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:08,555::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:39:08,561::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000211704 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) (1024) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:08,562::__init__::225::IOProcess::(_processLogs) (1024) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:08,563::__init__::225::IOProcess::(_processLogs) (1024) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:08,564::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:08,564::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:08,564::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:39:08,564::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) (1025) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) (1025) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) (1025) Building response >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) (1025) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:08,565::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:08,566::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:39:08,566::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:08,566::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:08,566::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:08,566::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) (1026) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) (1026) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) (1026) Building response >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) (1026) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:39:08,567::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:08,568::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:08,568::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:08,568::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:08,568::__init__::225::IOProcess::(_processLogs) (1027) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:08,568::__init__::225::IOProcess::(_processLogs) (1027) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:08,568::__init__::225::IOProcess::(_processLogs) (1027) Building response >Thread-17::DEBUG::2014-08-22 09:39:08,569::__init__::225::IOProcess::(_processLogs) (1027) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:08,569::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:08,569::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:39:10,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:10,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121622 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:12,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:12,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119426 s, 85.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:13,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`9749066e-98dc-4502-b3e2-c6da80626aec`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:13,703::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:39:13,703::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000184045', 'lastCheck': '5.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000211704', 'lastCheck': '5.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:13,703::task::1191::Storage.TaskManager.Task::(prepare) Task=`9749066e-98dc-4502-b3e2-c6da80626aec`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000184045', 'lastCheck': '5.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000211704', 'lastCheck': '5.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:13,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`9749066e-98dc-4502-b3e2-c6da80626aec`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:13,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:13,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:13,703::task::993::Storage.TaskManager.Task::(_decref) Task=`9749066e-98dc-4502-b3e2-c6da80626aec`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:14,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:14,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177774 s, 57.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:16,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-47::DEBUG::2014-08-22 09:39:16,675::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:39:16,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`a62a60e5-b002-4e51-8fab-0339f514d5ec`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:16,675::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:39:16,675::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:16,676::task::1191::Storage.TaskManager.Task::(prepare) Task=`a62a60e5-b002-4e51-8fab-0339f514d5ec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:16,676::task::595::Storage.TaskManager.Task::(_updateState) Task=`a62a60e5-b002-4e51-8fab-0339f514d5ec`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:16,676::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:16,676::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:16,676::task::993::Storage.TaskManager.Task::(_decref) Task=`a62a60e5-b002-4e51-8fab-0339f514d5ec`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:16,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117857 s, 86.9 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:39:16,710::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:39:16,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`59ead76a-5be5-4a9a-aba7-6f4430d4b0fa`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:39:16,712::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:39:16,713::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`09b43d38-07d7-4fed-82e6-5e70543b7c61`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:39:16,713::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:39:16,714::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:39:16,714::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`09b43d38-07d7-4fed-82e6-5e70543b7c61`::Granted request >Thread-46::DEBUG::2014-08-22 09:39:16,716::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`59ead76a-5be5-4a9a-aba7-6f4430d4b0fa`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:39:16,716::task::993::Storage.TaskManager.Task::(_decref) Task=`59ead76a-5be5-4a9a-aba7-6f4430d4b0fa`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:39:16,725::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:16,726::task::1191::Storage.TaskManager.Task::(prepare) Task=`59ead76a-5be5-4a9a-aba7-6f4430d4b0fa`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:16,726::task::595::Storage.TaskManager.Task::(_updateState) Task=`59ead76a-5be5-4a9a-aba7-6f4430d4b0fa`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:39:16,726::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:39:16,727::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:39:16,728::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:39:16,728::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:39:16,728::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:39:16,729::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:39:16,729::task::993::Storage.TaskManager.Task::(_decref) Task=`59ead76a-5be5-4a9a-aba7-6f4430d4b0fa`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:39:17,838::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:17,839::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:17,840::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:17,841::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:17,841::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:17,844::__init__::225::IOProcess::(_processLogs) (1028) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:17,845::__init__::225::IOProcess::(_processLogs) (1028) Got request for method 'statvfs' >Thread-21::DEBUG::2014-08-22 09:39:17,846::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:17,852::__init__::225::IOProcess::(_processLogs) (1028) Building response >Thread-17::DEBUG::2014-08-22 09:39:17,854::__init__::225::IOProcess::(_processLogs) (1028) Queuing response >Thread-21::DEBUG::2014-08-22 09:39:17,855::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000229289 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:17,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:17,856::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:17,856::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:17,856::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:17,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:17,857::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:17,857::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:17,857::__init__::225::IOProcess::(_processLogs) (1029) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:17,857::__init__::225::IOProcess::(_processLogs) (1029) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:17,858::__init__::225::IOProcess::(_processLogs) (1029) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:17,858::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:17,858::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) (1030) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:18,569::__init__::225::IOProcess::(_processLogs) (1030) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:18,570::__init__::225::IOProcess::(_processLogs) (1030) Queuing response >Thread-22::DEBUG::2014-08-22 09:39:18,570::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:18,570::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:18,576::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:39:18,577::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000235062 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:18,577::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:18,577::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:18,577::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:18,577::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:18,577::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:18,578::__init__::225::IOProcess::(_processLogs) (1031) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:18,578::__init__::225::IOProcess::(_processLogs) (1031) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:18,578::__init__::225::IOProcess::(_processLogs) (1031) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:18,579::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:18,579::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:18,579::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:39:18,579::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:18,579::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:18,579::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:18,580::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:18,580::__init__::225::IOProcess::(_processLogs) (1032) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:18,580::__init__::225::IOProcess::(_processLogs) (1032) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:18,580::__init__::225::IOProcess::(_processLogs) (1032) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:18,580::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:18,580::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:18,581::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:39:18,581::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:18,581::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:18,581::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) (1033) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) (1033) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) (1033) Building response >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) (1033) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:39:18,582::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:18,583::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:18,583::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:18,583::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:18,583::__init__::225::IOProcess::(_processLogs) (1034) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:18,583::__init__::225::IOProcess::(_processLogs) (1034) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:18,584::__init__::225::IOProcess::(_processLogs) (1034) Building response >Thread-17::DEBUG::2014-08-22 09:39:18,584::__init__::225::IOProcess::(_processLogs) (1034) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:18,584::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:18,584::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:39:18,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:18,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117314 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:20,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:20,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121383 s, 84.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:22,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:22,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116988 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:24,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:24,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011963 s, 85.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:26,778::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:39:26,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`ade6390b-d069-4d3a-8c9e-5c5ddf581721`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:26,779::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:39:26,779::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:26,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`ade6390b-d069-4d3a-8c9e-5c5ddf581721`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:26,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`ade6390b-d069-4d3a-8c9e-5c5ddf581721`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:26,780::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:26,780::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:26,780::task::993::Storage.TaskManager.Task::(_decref) Task=`ade6390b-d069-4d3a-8c9e-5c5ddf581721`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:39:26,804::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:39:26,804::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bc6cec5-0d11-4686-92bf-fb65dc69652e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:39:26,804::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:39:26,805::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`32936ce8-a7ed-4116-88c5-ee100465b49d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:39:26,805::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:39:26,805::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:39:26,805::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`32936ce8-a7ed-4116-88c5-ee100465b49d`::Granted request >Thread-46::DEBUG::2014-08-22 09:39:26,805::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4bc6cec5-0d11-4686-92bf-fb65dc69652e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:39:26,805::task::993::Storage.TaskManager.Task::(_decref) Task=`4bc6cec5-0d11-4686-92bf-fb65dc69652e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:39:26,807::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:26,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bc6cec5-0d11-4686-92bf-fb65dc69652e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:26,808::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bc6cec5-0d11-4686-92bf-fb65dc69652e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:39:26,808::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:39:26,808::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:39:26,808::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:39:26,808::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:39:26,808::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:39:26,808::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:39:26,808::task::993::Storage.TaskManager.Task::(_decref) Task=`4bc6cec5-0d11-4686-92bf-fb65dc69652e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:26,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:26,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011863 s, 86.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) (1035) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:27,859::__init__::225::IOProcess::(_processLogs) (1035) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:27,860::__init__::225::IOProcess::(_processLogs) (1035) Queuing response >Thread-21::DEBUG::2014-08-22 09:39:27,861::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:27,861::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:27,867::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:39:27,868::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000211949 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) (1036) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) (1036) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) (1036) Building response >Thread-17::DEBUG::2014-08-22 09:39:27,869::__init__::225::IOProcess::(_processLogs) (1036) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:27,870::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) (1037) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:28,584::__init__::225::IOProcess::(_processLogs) (1037) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:28,585::__init__::225::IOProcess::(_processLogs) (1037) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:28,585::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:39:28,585::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:28,586::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:39:28,592::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000240349 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:28,592::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:28,592::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:28,593::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:28,593::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:28,593::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:28,593::__init__::225::IOProcess::(_processLogs) (1038) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:28,593::__init__::225::IOProcess::(_processLogs) (1038) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:28,594::__init__::225::IOProcess::(_processLogs) (1038) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:28,594::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:28,594::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:28,594::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:39:28,595::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:28,595::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:28,595::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:28,595::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:28,595::__init__::225::IOProcess::(_processLogs) (1039) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) (1039) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) (1039) Building response >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) (1039) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:39:28,596::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) (1040) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) (1040) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) (1040) Building response >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) (1040) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:28,597::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:28,598::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:39:28,598::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:28,598::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:28,598::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) (1041) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) (1041) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) (1041) Building response >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) (1041) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:28,599::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:39:28,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`cceaa84b-427b-438c-9ae1-eb8ecc621928`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:28,875::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:39:28,875::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000211949', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000240349', 'lastCheck': '0.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:28,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`cceaa84b-427b-438c-9ae1-eb8ecc621928`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000211949', 'lastCheck': '1.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000240349', 'lastCheck': '0.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:28,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`cceaa84b-427b-438c-9ae1-eb8ecc621928`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:28,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:28,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:28,876::task::993::Storage.TaskManager.Task::(_decref) Task=`cceaa84b-427b-438c-9ae1-eb8ecc621928`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:28,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:29,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119742 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:31,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:31,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118724 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:33,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:33,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117364 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:35,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:35,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115672 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:36,874::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:39:36,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3ece712-6425-40b1-92f8-8e0204e2c3d5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:36,874::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:39:36,874::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:36,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3ece712-6425-40b1-92f8-8e0204e2c3d5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:36,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3ece712-6425-40b1-92f8-8e0204e2c3d5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:36,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:36,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:36,875::task::993::Storage.TaskManager.Task::(_decref) Task=`f3ece712-6425-40b1-92f8-8e0204e2c3d5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:39:36,913::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:39:36,913::task::595::Storage.TaskManager.Task::(_updateState) Task=`1054c78c-0f6a-4487-8244-241f015906c7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:39:36,913::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:39:36,913::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5b570735-3a80-4f94-a919-8e7c49eeb042`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:39:36,913::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:39:36,913::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:39:36,914::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5b570735-3a80-4f94-a919-8e7c49eeb042`::Granted request >Thread-46::DEBUG::2014-08-22 09:39:36,914::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1054c78c-0f6a-4487-8244-241f015906c7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:39:36,914::task::993::Storage.TaskManager.Task::(_decref) Task=`1054c78c-0f6a-4487-8244-241f015906c7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:39:36,916::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:36,916::task::1191::Storage.TaskManager.Task::(prepare) Task=`1054c78c-0f6a-4487-8244-241f015906c7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:36,916::task::595::Storage.TaskManager.Task::(_updateState) Task=`1054c78c-0f6a-4487-8244-241f015906c7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:39:36,916::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:39:36,916::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:39:36,916::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:39:36,916::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:39:36,916::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:39:36,916::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:39:36,917::task::993::Storage.TaskManager.Task::(_decref) Task=`1054c78c-0f6a-4487-8244-241f015906c7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:37,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:37,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119156 s, 85.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:37,871::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:37,872::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:37,872::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:37,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:37,872::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:37,872::__init__::225::IOProcess::(_processLogs) (1042) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:37,872::__init__::225::IOProcess::(_processLogs) (1042) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:37,873::__init__::225::IOProcess::(_processLogs) (1042) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:37,873::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:39:37,873::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:37,874::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:39:37,880::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000150913 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) (1043) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:37,881::__init__::225::IOProcess::(_processLogs) (1043) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:37,882::__init__::225::IOProcess::(_processLogs) (1043) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:37,883::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:37,883::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:38,599::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:38,599::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:38,599::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:38,599::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:38,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:38,599::__init__::225::IOProcess::(_processLogs) (1044) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:38,600::__init__::225::IOProcess::(_processLogs) (1044) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:38,600::__init__::225::IOProcess::(_processLogs) (1044) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:38,600::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:39:38,601::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:38,601::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:39:38,607::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216669 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:38,607::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:38,608::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:38,608::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:38,608::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:38,608::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:38,608::__init__::225::IOProcess::(_processLogs) (1045) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:38,608::__init__::225::IOProcess::(_processLogs) (1045) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:38,609::__init__::225::IOProcess::(_processLogs) (1045) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:38,609::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:38,609::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:38,610::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:39:38,610::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:38,610::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:38,610::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:38,610::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) (1046) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) (1046) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) (1046) Building response >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) (1046) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:39:38,611::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) (1047) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) (1047) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) (1047) Building response >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) (1047) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:38,612::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:38,613::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:38,613::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:39:38,613::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:38,613::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:38,613::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) (1048) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) (1048) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) (1048) Building response >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) (1048) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:38,614::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:39:39,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:39,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116937 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:41,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:41,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116165 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:43,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:43,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117026 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:44,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`934119ac-035b-4373-a905-ce16a49db765`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:44,169::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:39:44,170::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000150913', 'lastCheck': '6.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216669', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:44,170::task::1191::Storage.TaskManager.Task::(prepare) Task=`934119ac-035b-4373-a905-ce16a49db765`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000150913', 'lastCheck': '6.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216669', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:44,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`934119ac-035b-4373-a905-ce16a49db765`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:44,170::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:44,170::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:44,170::task::993::Storage.TaskManager.Task::(_decref) Task=`934119ac-035b-4373-a905-ce16a49db765`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:45,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:45,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121107 s, 84.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:46,972::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:39:46,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`171844a0-813c-4b3c-9def-005ed27c1fad`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:46,973::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:39:46,973::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:46,973::task::1191::Storage.TaskManager.Task::(prepare) Task=`171844a0-813c-4b3c-9def-005ed27c1fad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:46,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`171844a0-813c-4b3c-9def-005ed27c1fad`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:46,973::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:46,974::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:46,974::task::993::Storage.TaskManager.Task::(_decref) Task=`171844a0-813c-4b3c-9def-005ed27c1fad`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:39:47,000::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:39:47,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f756d01-1bb1-4011-97b7-ef0521640929`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:39:47,000::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:39:47,000::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0dfa19ed-bb6e-419f-840a-c47f5b4fb7a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:39:47,001::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:39:47,001::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:39:47,001::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0dfa19ed-bb6e-419f-840a-c47f5b4fb7a6`::Granted request >Thread-46::DEBUG::2014-08-22 09:39:47,001::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1f756d01-1bb1-4011-97b7-ef0521640929`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:39:47,001::task::993::Storage.TaskManager.Task::(_decref) Task=`1f756d01-1bb1-4011-97b7-ef0521640929`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:39:47,003::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:47,003::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f756d01-1bb1-4011-97b7-ef0521640929`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:47,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f756d01-1bb1-4011-97b7-ef0521640929`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:39:47,003::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:39:47,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:39:47,004::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:39:47,004::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:39:47,004::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:39:47,004::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:39:47,004::task::993::Storage.TaskManager.Task::(_decref) Task=`1f756d01-1bb1-4011-97b7-ef0521640929`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:47,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:47,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117927 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:47,883::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:47,883::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:47,884::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:47,884::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:47,884::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:47,884::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:47,884::__init__::225::IOProcess::(_processLogs) (1049) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:47,884::__init__::225::IOProcess::(_processLogs) (1049) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:47,885::__init__::225::IOProcess::(_processLogs) DEBUG|(1049) Building response >Thread-21::DEBUG::2014-08-22 09:39:47,885::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:47,885::__init__::225::IOProcess::(_processLogs) (1049) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:47,891::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:47,891::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:39:47,891::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000246654 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:47,892::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:47,892::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:47,892::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:47,892::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:47,892::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) (1050) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) (1050) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) (1050) Building response >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) (1050) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:47,893::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:47,894::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:48,614::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:48,614::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:48,615::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:48,615::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:48,616::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:39:48,616::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:48,617::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:48,623::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:39:48,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000196356 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:48,624::__init__::225::IOProcess::(_processLogs) (1051) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:48,624::__init__::225::IOProcess::(_processLogs) (1051) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:48,624::__init__::225::IOProcess::(_processLogs) (1051) Building response >Thread-17::DEBUG::2014-08-22 09:39:48,624::__init__::225::IOProcess::(_processLogs) (1051) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:48,625::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:48,626::__init__::225::IOProcess::(_processLogs) (1052) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:48,626::__init__::225::IOProcess::(_processLogs) (1052) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:48,626::__init__::225::IOProcess::(_processLogs) DEBUG|(1052) Building response >Thread-17::DEBUG::2014-08-22 09:39:48,627::__init__::225::IOProcess::(_processLogs) (1052) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:48,627::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:48,627::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:48,627::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:48,627::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:39:48,627::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) (1053) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) (1053) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:48,628::__init__::225::IOProcess::(_processLogs) (1053) Building response >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) (1053) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:48,629::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) (1054) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) (1054) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) (1054) Building response >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) (1054) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:48,630::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:48,631::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:48,631::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:39:48,631::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:48,631::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:48,631::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) (1055) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) (1055) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) (1055) Building response >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) (1055) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:48,632::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:39:49,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:49,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120117 s, 85.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:51,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:51,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116716 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:53,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:53,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114785 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:39:55,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:55,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114973 s, 89.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:39:57,052::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:39:57,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`90b9540d-c8db-4363-baa7-23de7a135eed`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:57,052::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:39:57,053::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:57,053::task::1191::Storage.TaskManager.Task::(prepare) Task=`90b9540d-c8db-4363-baa7-23de7a135eed`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:39:57,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`90b9540d-c8db-4363-baa7-23de7a135eed`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:57,053::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:57,053::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:57,053::task::993::Storage.TaskManager.Task::(_decref) Task=`90b9540d-c8db-4363-baa7-23de7a135eed`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:39:57,083::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:39:57,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`51cb6196-e77c-4a9d-9121-89983b31022e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:39:57,084::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:39:57,084::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3368f173-0223-4def-b5aa-663978d89f07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:39:57,084::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:39:57,084::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:39:57,084::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3368f173-0223-4def-b5aa-663978d89f07`::Granted request >Thread-46::DEBUG::2014-08-22 09:39:57,084::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`51cb6196-e77c-4a9d-9121-89983b31022e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:39:57,084::task::993::Storage.TaskManager.Task::(_decref) Task=`51cb6196-e77c-4a9d-9121-89983b31022e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:39:57,086::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:57,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`51cb6196-e77c-4a9d-9121-89983b31022e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:39:57,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`51cb6196-e77c-4a9d-9121-89983b31022e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:39:57,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:39:57,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:39:57,087::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:39:57,087::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:39:57,087::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:39:57,087::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:39:57,087::task::993::Storage.TaskManager.Task::(_decref) Task=`51cb6196-e77c-4a9d-9121-89983b31022e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:57,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:57,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115199 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:57,895::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:57,895::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:57,895::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:57,895::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:57,896::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:57,896::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:57,896::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:57,896::__init__::225::IOProcess::(_processLogs) (1056) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:57,896::__init__::225::IOProcess::(_processLogs) (1056) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:57,896::__init__::225::IOProcess::(_processLogs) DEBUG|(1056) Building response >Thread-21::DEBUG::2014-08-22 09:39:57,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:57,897::__init__::225::IOProcess::(_processLogs) (1056) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:57,903::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:39:57,904::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000185956 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:57,905::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) (1057) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:57,906::__init__::225::IOProcess::(_processLogs) (1057) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:57,907::__init__::225::IOProcess::(_processLogs) DEBUG|(1057) Building response >Thread-17::DEBUG::2014-08-22 09:39:57,908::__init__::225::IOProcess::(_processLogs) (1057) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:57,908::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:57,908::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:58,632::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:58,632::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:58,632::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:58,632::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:58,632::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:58,633::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:58,633::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:58,633::__init__::225::IOProcess::(_processLogs) (1058) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:58,633::__init__::225::IOProcess::(_processLogs) (1058) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:58,633::__init__::225::IOProcess::(_processLogs) DEBUG|(1058) Building response >Thread-17::DEBUG::2014-08-22 09:39:58,634::__init__::225::IOProcess::(_processLogs) (1058) Queuing response >Thread-22::DEBUG::2014-08-22 09:39:58,634::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:39:58,634::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:58,640::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:39:58,641::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000271673 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:58,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:58,643::__init__::225::IOProcess::(_processLogs) (1059) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:39:58,643::__init__::225::IOProcess::(_processLogs) (1059) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:39:58,643::__init__::225::IOProcess::(_processLogs) DEBUG|(1059) Building response >Thread-17::DEBUG::2014-08-22 09:39:58,644::__init__::225::IOProcess::(_processLogs) (1059) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:58,644::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:58,644::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:39:58,644::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:58,644::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:39:58,645::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:58,645::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:58,645::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:58,645::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:58,645::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:58,645::__init__::225::IOProcess::(_processLogs) (1060) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) (1060) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) (1060) Building response >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) (1060) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:58,646::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) (1061) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) (1061) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) (1061) Building response >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) (1061) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:58,647::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:39:58,648::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:39:58,648::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:39:58,648::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:39:58,649::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:39:58,649::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:39:58,649::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:39:58,649::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) (1062) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) (1062) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) (1062) Building response >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) (1062) Queuing response >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:39:58,650::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:39:59,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`3413f41b-b449-41ef-bc19-5c92bdfa4f35`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:39:59,393::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:39:59,394::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000185956', 'lastCheck': '1.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000271673', 'lastCheck': '0.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:59,394::task::1191::Storage.TaskManager.Task::(prepare) Task=`3413f41b-b449-41ef-bc19-5c92bdfa4f35`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000185956', 'lastCheck': '1.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000271673', 'lastCheck': '0.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:39:59,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`3413f41b-b449-41ef-bc19-5c92bdfa4f35`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:39:59,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:39:59,394::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:39:59,394::task::993::Storage.TaskManager.Task::(_decref) Task=`3413f41b-b449-41ef-bc19-5c92bdfa4f35`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:39:59,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:39:59,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121122 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:01,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:01,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118555 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:03,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:03,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114745 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:05,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:05,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119359 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:07,180::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:40:07,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f2166d9-76dc-4241-9691-3e8a4901d0a4`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:07,180::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:40:07,181::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:07,181::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f2166d9-76dc-4241-9691-3e8a4901d0a4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:07,181::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f2166d9-76dc-4241-9691-3e8a4901d0a4`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:07,181::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:07,181::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:07,181::task::993::Storage.TaskManager.Task::(_decref) Task=`8f2166d9-76dc-4241-9691-3e8a4901d0a4`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:40:07,345::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:40:07,345::task::595::Storage.TaskManager.Task::(_updateState) Task=`864098e8-2868-43c6-845c-029cc194f600`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:40:07,345::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:40:07,346::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9750856b-75fa-40e5-bdbb-6308d6d6025d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:40:07,346::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:40:07,346::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:40:07,346::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9750856b-75fa-40e5-bdbb-6308d6d6025d`::Granted request >Thread-46::DEBUG::2014-08-22 09:40:07,346::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`864098e8-2868-43c6-845c-029cc194f600`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:40:07,346::task::993::Storage.TaskManager.Task::(_decref) Task=`864098e8-2868-43c6-845c-029cc194f600`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:40:07,348::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:07,348::task::1191::Storage.TaskManager.Task::(prepare) Task=`864098e8-2868-43c6-845c-029cc194f600`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:07,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`864098e8-2868-43c6-845c-029cc194f600`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:40:07,349::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:40:07,349::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:40:07,349::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:40:07,349::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:40:07,349::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:40:07,349::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:40:07,349::task::993::Storage.TaskManager.Task::(_decref) Task=`864098e8-2868-43c6-845c-029cc194f600`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:07,908::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:07,908::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:07,909::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:07,909::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:07,909::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:07,909::__init__::225::IOProcess::(_processLogs) (1063) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:07,909::__init__::225::IOProcess::(_processLogs) (1063) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:07,909::__init__::225::IOProcess::(_processLogs) (1063) Queuing response >Thread-21::DEBUG::2014-08-22 09:40:07,910::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:07,910::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:07,916::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:40:07,917::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000206659 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) (1064) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:07,918::__init__::225::IOProcess::(_processLogs) (1064) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:07,919::__init__::225::IOProcess::(_processLogs) (1064) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:07,920::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:07,920::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:40:07,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:07,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227113 s, 45.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) (1065) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:08,650::__init__::225::IOProcess::(_processLogs) (1065) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:08,651::__init__::225::IOProcess::(_processLogs) (1065) Queuing response >Thread-22::DEBUG::2014-08-22 09:40:08,652::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:08,652::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:08,658::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:40:08,659::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000179402 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:08,659::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:08,659::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:08,660::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:08,660::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:08,660::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:08,660::__init__::225::IOProcess::(_processLogs) (1066) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:08,660::__init__::225::IOProcess::(_processLogs) (1066) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:08,661::__init__::225::IOProcess::(_processLogs) (1066) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:08,661::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:08,661::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:08,661::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:40:08,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:08,662::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:08,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:08,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:08,662::__init__::225::IOProcess::(_processLogs) (1067) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) (1067) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) (1067) Building response >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) (1067) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:40:08,663::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) (1068) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) (1068) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) (1068) Building response >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) (1068) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:08,664::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:08,665::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:40:08,665::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:08,665::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:08,665::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:08,665::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:08,666::__init__::225::IOProcess::(_processLogs) (1069) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:08,666::__init__::225::IOProcess::(_processLogs) (1069) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:08,666::__init__::225::IOProcess::(_processLogs) (1069) Building response >Thread-17::DEBUG::2014-08-22 09:40:08,666::__init__::225::IOProcess::(_processLogs) (1069) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:08,666::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:08,666::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:40:09,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:10,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011904 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:12,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:12,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117014 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:14,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:14,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117394 s, 87.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:14,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`de4c352f-790a-4c71-9afa-5d6b83108707`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:14,652::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:40:14,652::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206659', 'lastCheck': '6.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000179402', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:40:14,652::task::1191::Storage.TaskManager.Task::(prepare) Task=`de4c352f-790a-4c71-9afa-5d6b83108707`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206659', 'lastCheck': '6.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000179402', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:40:14,652::task::595::Storage.TaskManager.Task::(_updateState) Task=`de4c352f-790a-4c71-9afa-5d6b83108707`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:14,652::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:14,652::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:14,652::task::993::Storage.TaskManager.Task::(_decref) Task=`de4c352f-790a-4c71-9afa-5d6b83108707`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:40:16,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:16,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118625 s, 86.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:17,451::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:40:17,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`b70b7706-89c4-410f-a6c0-2d7f6e3a4f93`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:17,451::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:40:17,451::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:17,452::task::1191::Storage.TaskManager.Task::(prepare) Task=`b70b7706-89c4-410f-a6c0-2d7f6e3a4f93`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:17,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`b70b7706-89c4-410f-a6c0-2d7f6e3a4f93`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:17,452::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:17,452::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:17,452::task::993::Storage.TaskManager.Task::(_decref) Task=`b70b7706-89c4-410f-a6c0-2d7f6e3a4f93`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:40:17,485::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:40:17,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`40359861-180b-439c-8b83-28d827c28564`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:40:17,486::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:40:17,486::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5044a09a-2dc7-4657-bed9-f42ef3ed51ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:40:17,486::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:40:17,486::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:40:17,486::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5044a09a-2dc7-4657-bed9-f42ef3ed51ec`::Granted request >Thread-46::DEBUG::2014-08-22 09:40:17,486::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`40359861-180b-439c-8b83-28d827c28564`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:40:17,487::task::993::Storage.TaskManager.Task::(_decref) Task=`40359861-180b-439c-8b83-28d827c28564`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:40:17,488::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:17,489::task::1191::Storage.TaskManager.Task::(prepare) Task=`40359861-180b-439c-8b83-28d827c28564`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:17,489::task::595::Storage.TaskManager.Task::(_updateState) Task=`40359861-180b-439c-8b83-28d827c28564`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:40:17,489::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:40:17,489::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:40:17,489::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:40:17,489::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:40:17,489::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:40:17,489::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:40:17,489::task::993::Storage.TaskManager.Task::(_decref) Task=`40359861-180b-439c-8b83-28d827c28564`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:17,920::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:17,920::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:17,920::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:17,921::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:17,921::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:17,921::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:17,921::__init__::225::IOProcess::(_processLogs) (1070) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:17,921::__init__::225::IOProcess::(_processLogs) (1070) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:17,921::__init__::225::IOProcess::(_processLogs) (1070) Queuing response >Thread-21::DEBUG::2014-08-22 09:40:17,922::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:17,922::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:17,928::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:40:17,929::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000166521 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) (1071) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:17,930::__init__::225::IOProcess::(_processLogs) (1071) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:17,931::__init__::225::IOProcess::(_processLogs) (1071) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:17,932::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:17,932::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:40:18,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:18,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120369 s, 85.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) (1072) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:18,670::__init__::225::IOProcess::(_processLogs) (1072) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:18,671::__init__::225::IOProcess::(_processLogs) DEBUG|(1072) Building response >Thread-22::DEBUG::2014-08-22 09:40:18,672::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:18,672::__init__::225::IOProcess::(_processLogs) (1072) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:18,678::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:40:18,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000205307 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:18,679::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:18,680::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:18,680::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:18,680::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:18,680::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:18,680::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:18,681::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:18,681::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:18,681::__init__::225::IOProcess::(_processLogs) (1073) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:18,681::__init__::225::IOProcess::(_processLogs) (1073) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:18,681::__init__::225::IOProcess::(_processLogs) DEBUG|(1073) Building response >Thread-17::DEBUG::2014-08-22 09:40:18,682::__init__::225::IOProcess::(_processLogs) (1073) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:18,682::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:18,682::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:18,683::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:18,683::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:40:18,683::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:18,683::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) (1074) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) (1074) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) (1074) Building response >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) (1074) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:18,684::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) (1075) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:18,685::__init__::225::IOProcess::(_processLogs) (1075) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) (1075) Building response >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) (1075) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:40:18,686::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:18,687::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:18,687::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:18,687::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:18,687::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:18,687::__init__::225::IOProcess::(_processLogs) (1076) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:18,687::__init__::225::IOProcess::(_processLogs) (1076) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:18,688::__init__::225::IOProcess::(_processLogs) (1076) Building response >Thread-17::DEBUG::2014-08-22 09:40:18,688::__init__::225::IOProcess::(_processLogs) (1076) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:18,688::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:18,688::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:40:20,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:20,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119941 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:22,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:22,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116386 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:24,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:24,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116104 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:26,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:26,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169178 s, 60.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:27,545::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:40:27,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`af152e49-eb6c-45c4-ab32-e02179464c2b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:27,545::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:40:27,545::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:27,546::task::1191::Storage.TaskManager.Task::(prepare) Task=`af152e49-eb6c-45c4-ab32-e02179464c2b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:27,546::task::595::Storage.TaskManager.Task::(_updateState) Task=`af152e49-eb6c-45c4-ab32-e02179464c2b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:27,546::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:27,546::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:27,546::task::993::Storage.TaskManager.Task::(_decref) Task=`af152e49-eb6c-45c4-ab32-e02179464c2b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:40:27,590::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:40:27,590::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2f1df10-9941-400e-827c-5c174e66a594`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:40:27,590::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:40:27,590::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4fd34207-4acc-4599-9949-5046a6b04be5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:40:27,590::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:40:27,590::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:40:27,590::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4fd34207-4acc-4599-9949-5046a6b04be5`::Granted request >Thread-46::DEBUG::2014-08-22 09:40:27,591::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b2f1df10-9941-400e-827c-5c174e66a594`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:40:27,591::task::993::Storage.TaskManager.Task::(_decref) Task=`b2f1df10-9941-400e-827c-5c174e66a594`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:40:27,593::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:27,593::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2f1df10-9941-400e-827c-5c174e66a594`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:27,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2f1df10-9941-400e-827c-5c174e66a594`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:40:27,593::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:40:27,593::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:40:27,593::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:40:27,593::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:40:27,593::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:40:27,593::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:40:27,594::task::993::Storage.TaskManager.Task::(_decref) Task=`b2f1df10-9941-400e-827c-5c174e66a594`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:27,932::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:27,932::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) (1077) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:27,933::__init__::225::IOProcess::(_processLogs) (1077) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:27,934::__init__::225::IOProcess::(_processLogs) (1077) Queuing response >Thread-21::DEBUG::2014-08-22 09:40:27,934::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:27,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:27,941::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:40:27,942::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196494 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:27,942::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:27,942::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:27,942::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:27,942::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:27,943::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:27,943::__init__::225::IOProcess::(_processLogs) (1078) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:27,943::__init__::225::IOProcess::(_processLogs) (1078) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:27,943::__init__::225::IOProcess::(_processLogs) (1078) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:27,944::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:27,944::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:40:28,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:28,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120695 s, 84.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) (1079) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:28,688::__init__::225::IOProcess::(_processLogs) (1079) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:28,689::__init__::225::IOProcess::(_processLogs) (1079) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:28,690::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:40:28,690::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:28,690::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:40:28,697::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000182272 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:28,697::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:28,697::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:28,697::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:28,698::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:28,698::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:28,698::__init__::225::IOProcess::(_processLogs) (1080) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:28,698::__init__::225::IOProcess::(_processLogs) (1080) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:28,698::__init__::225::IOProcess::(_processLogs) (1080) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:28,699::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:28,699::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:28,699::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:40:28,700::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:28,700::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:28,700::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:28,700::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:28,700::__init__::225::IOProcess::(_processLogs) (1081) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) (1081) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) (1081) Building response >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) (1081) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:40:28,701::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) (1082) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) (1082) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) (1082) Building response >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) (1082) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:28,702::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:28,703::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:28,703::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:40:28,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:28,703::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) (1083) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) (1083) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) (1083) Building response >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) (1083) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:28,704::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:28,705::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:40:29,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9b59f03-f2e0-43c4-96cd-1ad5213a713e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:29,836::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:40:29,836::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196494', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182272', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:40:29,836::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9b59f03-f2e0-43c4-96cd-1ad5213a713e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196494', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182272', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:40:29,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9b59f03-f2e0-43c4-96cd-1ad5213a713e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:29,836::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:29,836::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:29,836::task::993::Storage.TaskManager.Task::(_decref) Task=`c9b59f03-f2e0-43c4-96cd-1ad5213a713e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:40:30,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:30,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120874 s, 84.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:32,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:32,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178842 s, 57.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:34,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:34,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114651 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:36,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:36,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115973 s, 88.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:37,645::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:40:37,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa880705-74b7-452a-bb00-a67e0aaa4396`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:37,647::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:40:37,649::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:37,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`aa880705-74b7-452a-bb00-a67e0aaa4396`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:37,650::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa880705-74b7-452a-bb00-a67e0aaa4396`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:37,651::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:37,651::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:37,652::task::993::Storage.TaskManager.Task::(_decref) Task=`aa880705-74b7-452a-bb00-a67e0aaa4396`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:40:37,894::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:40:37,894::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e319c03-b351-4b93-925a-a7f34a6708b3`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:40:37,894::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:40:37,895::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`64ba8bcc-f7e4-4b83-b93f-5bb680b9ea01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:40:37,895::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:40:37,895::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:40:37,895::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`64ba8bcc-f7e4-4b83-b93f-5bb680b9ea01`::Granted request >Thread-46::DEBUG::2014-08-22 09:40:37,895::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6e319c03-b351-4b93-925a-a7f34a6708b3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:40:37,895::task::993::Storage.TaskManager.Task::(_decref) Task=`6e319c03-b351-4b93-925a-a7f34a6708b3`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:40:37,897::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:37,897::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e319c03-b351-4b93-925a-a7f34a6708b3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:37,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e319c03-b351-4b93-925a-a7f34a6708b3`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:40:37,897::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:40:37,898::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:40:37,898::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:40:37,898::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:40:37,898::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:40:37,898::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:40:37,898::task::993::Storage.TaskManager.Task::(_decref) Task=`6e319c03-b351-4b93-925a-a7f34a6708b3`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) (1084) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:37,945::__init__::225::IOProcess::(_processLogs) (1084) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:37,946::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1084) Building response >Thread-21::DEBUG::2014-08-22 09:40:37,946::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:37,946::__init__::225::IOProcess::(_processLogs) (1084) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:37,953::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:40:37,953::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000195052 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:37,954::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:37,955::__init__::225::IOProcess::(_processLogs) (1085) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:37,956::__init__::225::IOProcess::(_processLogs) (1085) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:37,956::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1085) Building response >Thread-17::DEBUG::2014-08-22 09:40:37,957::__init__::225::IOProcess::(_processLogs) (1085) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:37,957::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:37,957::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:38,704::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:38,704::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:38,705::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:38,705::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:38,706::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:40:38,706::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:38,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:38,713::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:40:38,713::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000236672 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:38,714::__init__::225::IOProcess::(_processLogs) (1086) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:38,714::__init__::225::IOProcess::(_processLogs) (1086) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:38,714::__init__::225::IOProcess::(_processLogs) (1086) Building response >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) (1086) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:38,715::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:38,716::__init__::225::IOProcess::(_processLogs) (1087) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:38,716::__init__::225::IOProcess::(_processLogs) (1087) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:38,716::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1087) Building response >Thread-17::DEBUG::2014-08-22 09:40:38,717::__init__::225::IOProcess::(_processLogs) (1087) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:38,717::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:38,717::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:38,717::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:38,717::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:40:38,717::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:38,718::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:38,718::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:38,718::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:38,718::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:38,718::__init__::225::IOProcess::(_processLogs) (1088) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:38,718::__init__::225::IOProcess::(_processLogs) (1088) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) (1088) Building response >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) (1088) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:40:38,719::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) (1089) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) (1089) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) (1089) Building response >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) (1089) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:38,720::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:38,721::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:38,721::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:40:38,721::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:38,721::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:38,721::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) (1090) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) (1090) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) (1090) Building response >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) (1090) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:38,722::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:40:38,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:38,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120815 s, 84.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:40,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:40,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116204 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:42,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:42,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118421 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:44,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:44,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116396 s, 88.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:45,030::task::595::Storage.TaskManager.Task::(_updateState) Task=`43c532b8-c729-4498-abfe-6ae128fae03d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:45,030::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:40:45,030::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000195052', 'lastCheck': '7.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000236672', 'lastCheck': '6.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:40:45,030::task::1191::Storage.TaskManager.Task::(prepare) Task=`43c532b8-c729-4498-abfe-6ae128fae03d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000195052', 'lastCheck': '7.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000236672', 'lastCheck': '6.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:40:45,030::task::595::Storage.TaskManager.Task::(_updateState) Task=`43c532b8-c729-4498-abfe-6ae128fae03d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:45,030::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:45,030::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:45,031::task::993::Storage.TaskManager.Task::(_decref) Task=`43c532b8-c729-4498-abfe-6ae128fae03d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:40:46,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:47,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116919 s, 87.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:40:47,954::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:40:47,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`bde0105c-66d3-4d7a-8c30-8a55e96df8fb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:47,954::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:40:47,954::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:47,954::task::1191::Storage.TaskManager.Task::(prepare) Task=`bde0105c-66d3-4d7a-8c30-8a55e96df8fb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:47,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`bde0105c-66d3-4d7a-8c30-8a55e96df8fb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:47,955::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:47,955::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:47,955::task::993::Storage.TaskManager.Task::(_decref) Task=`bde0105c-66d3-4d7a-8c30-8a55e96df8fb`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:47,957::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:47,957::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) (1091) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:47,958::__init__::225::IOProcess::(_processLogs) (1091) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:47,959::__init__::225::IOProcess::(_processLogs) (1091) Queuing response >Thread-21::DEBUG::2014-08-22 09:40:47,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:47,959::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:47,966::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:40:47,966::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000207373 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:47,967::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:47,967::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:47,967::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:47,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:47,968::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:47,968::__init__::225::IOProcess::(_processLogs) (1092) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:47,968::__init__::225::IOProcess::(_processLogs) (1092) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:47,969::__init__::225::IOProcess::(_processLogs) (1092) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:47,969::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:47,969::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-46::DEBUG::2014-08-22 09:40:47,989::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:40:47,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`e559b690-444a-4303-a30a-787174f86649`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:40:47,989::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:40:47,990::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1daf4754-58d6-461b-b1fd-7e045eae0b92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:40:47,990::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:40:47,990::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:40:47,990::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1daf4754-58d6-461b-b1fd-7e045eae0b92`::Granted request >Thread-46::DEBUG::2014-08-22 09:40:47,991::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e559b690-444a-4303-a30a-787174f86649`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:40:47,991::task::993::Storage.TaskManager.Task::(_decref) Task=`e559b690-444a-4303-a30a-787174f86649`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:40:47,993::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:47,993::task::1191::Storage.TaskManager.Task::(prepare) Task=`e559b690-444a-4303-a30a-787174f86649`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:47,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`e559b690-444a-4303-a30a-787174f86649`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:40:47,993::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:40:47,993::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:40:47,993::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:40:47,994::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:40:47,994::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:40:47,994::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:40:47,994::task::993::Storage.TaskManager.Task::(_decref) Task=`e559b690-444a-4303-a30a-787174f86649`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:48,722::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:48,722::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:48,722::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:48,722::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:48,723::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:48,723::__init__::225::IOProcess::(_processLogs) (1093) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:48,723::__init__::225::IOProcess::(_processLogs) (1093) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:48,723::__init__::225::IOProcess::(_processLogs) (1093) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:48,724::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:40:48,724::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:48,724::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:40:48,730::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234907 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) (1094) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:48,731::__init__::225::IOProcess::(_processLogs) (1094) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:48,732::__init__::225::IOProcess::(_processLogs) (1094) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:48,732::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:48,733::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:48,733::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:40:48,733::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:48,733::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) (1095) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) (1095) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) (1095) Building response >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) (1095) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:40:48,734::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:48,735::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:48,735::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:48,735::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:48,735::__init__::225::IOProcess::(_processLogs) (1096) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:48,735::__init__::225::IOProcess::(_processLogs) (1096) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:48,735::__init__::225::IOProcess::(_processLogs) (1096) Building response >Thread-17::DEBUG::2014-08-22 09:40:48,736::__init__::225::IOProcess::(_processLogs) (1096) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:48,736::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:48,736::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:48,736::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:40:48,736::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:48,736::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) (1097) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) (1097) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) (1097) Building response >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) (1097) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:48,737::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:40:49,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:49,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123319 s, 83.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:51,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:51,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117579 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:53,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:53,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114731 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:55,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:55,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117127 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:40:57,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:57,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117081 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) (1098) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) (1098) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:57,970::__init__::225::IOProcess::(_processLogs) (1098) Building response >Thread-17::DEBUG::2014-08-22 09:40:57,971::__init__::225::IOProcess::(_processLogs) (1098) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:57,971::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:40:57,972::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:40:57,978::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000231973 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:57,978::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:40:57,978::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:57,979::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:57,979::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:57,979::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:57,979::__init__::225::IOProcess::(_processLogs) (1099) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:57,979::__init__::225::IOProcess::(_processLogs) (1099) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:57,979::__init__::225::IOProcess::(_processLogs) (1099) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:57,980::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:57,980::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:40:58,068::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:40:58,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f6117ef-1aed-41be-9ff9-ecd5fd686e44`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:40:58,068::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:40:58,068::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:58,069::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f6117ef-1aed-41be-9ff9-ecd5fd686e44`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:40:58,069::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f6117ef-1aed-41be-9ff9-ecd5fd686e44`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:40:58,069::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:40:58,069::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:40:58,069::task::993::Storage.TaskManager.Task::(_decref) Task=`5f6117ef-1aed-41be-9ff9-ecd5fd686e44`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:40:58,101::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:40:58,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`56f059a6-bf3f-4337-8605-16fd287c153e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:40:58,101::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:40:58,101::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9b5638e7-01ba-4162-85c2-320c1c5a51d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:40:58,101::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:40:58,101::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:40:58,101::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9b5638e7-01ba-4162-85c2-320c1c5a51d5`::Granted request >Thread-46::DEBUG::2014-08-22 09:40:58,102::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`56f059a6-bf3f-4337-8605-16fd287c153e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:40:58,102::task::993::Storage.TaskManager.Task::(_decref) Task=`56f059a6-bf3f-4337-8605-16fd287c153e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:40:58,104::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:58,104::task::1191::Storage.TaskManager.Task::(prepare) Task=`56f059a6-bf3f-4337-8605-16fd287c153e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:40:58,104::task::595::Storage.TaskManager.Task::(_updateState) Task=`56f059a6-bf3f-4337-8605-16fd287c153e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:40:58,104::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:40:58,104::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:40:58,104::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:40:58,104::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:40:58,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:40:58,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:40:58,105::task::993::Storage.TaskManager.Task::(_decref) Task=`56f059a6-bf3f-4337-8605-16fd287c153e`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:40:58,737::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:58,737::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) (1100) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) (1100) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:58,738::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1100) Building response >Thread-17::DEBUG::2014-08-22 09:40:58,739::__init__::225::IOProcess::(_processLogs) (1100) Queuing response >Thread-22::DEBUG::2014-08-22 09:40:58,739::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:40:58,739::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:58,746::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:40:58,746::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223064 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:40:58,747::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:58,747::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:40:58,747::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:58,747::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:58,747::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:58,748::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:58,748::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:58,748::__init__::225::IOProcess::(_processLogs) (1101) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:40:58,748::__init__::225::IOProcess::(_processLogs) (1101) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:40:58,748::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1101) Building response >Thread-17::DEBUG::2014-08-22 09:40:58,749::__init__::225::IOProcess::(_processLogs) (1101) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:58,749::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:58,749::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:40:58,749::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:58,750::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:40:58,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:58,750::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:58,750::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:58,750::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:58,750::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) (1102) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) (1102) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) (1102) Building response >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) (1102) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:58,751::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) (1103) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) (1103) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) (1103) Building response >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) (1103) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:58,752::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:58,753::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:40:58,753::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:40:58,753::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:40:58,753::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:40:58,753::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:40:58,754::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:40:58,754::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:40:58,754::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:40:58,754::__init__::225::IOProcess::(_processLogs) (1104) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:40:58,754::__init__::225::IOProcess::(_processLogs) (1104) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:40:58,755::__init__::225::IOProcess::(_processLogs) (1104) Building response >Thread-17::DEBUG::2014-08-22 09:40:58,755::__init__::225::IOProcess::(_processLogs) (1104) Queuing response >Thread-17::DEBUG::2014-08-22 09:40:58,755::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:40:58,755::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:40:59,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:40:59,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122977 s, 83.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:41:00,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`36b7c922-d59a-4c59-bf23-3c804024c028`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:00,202::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:41:00,202::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000231973', 'lastCheck': '2.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223064', 'lastCheck': '1.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:00,202::task::1191::Storage.TaskManager.Task::(prepare) Task=`36b7c922-d59a-4c59-bf23-3c804024c028`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000231973', 'lastCheck': '2.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223064', 'lastCheck': '1.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:00,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`36b7c922-d59a-4c59-bf23-3c804024c028`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:00,203::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:00,203::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:00,203::task::993::Storage.TaskManager.Task::(_decref) Task=`36b7c922-d59a-4c59-bf23-3c804024c028`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:41:01,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:01,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01179 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:03,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:03,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115476 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:05,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:05,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160356 s, 63.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:07,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:07,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121783 s, 84.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:07,981::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:07,982::__init__::225::IOProcess::(_processLogs) (1105) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:07,982::__init__::225::IOProcess::(_processLogs) (1105) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:07,982::__init__::225::IOProcess::(_processLogs) DEBUG|(1105) Building response >Thread-17::DEBUG::2014-08-22 09:41:07,983::__init__::225::IOProcess::(_processLogs) (1105) Queuing response >Thread-21::DEBUG::2014-08-22 09:41:07,983::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:07,984::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:07,989::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:41:07,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000212196 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:07,991::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:07,991::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:07,991::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:07,991::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:07,991::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:07,991::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:07,992::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:07,992::__init__::225::IOProcess::(_processLogs) (1106) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:07,992::__init__::225::IOProcess::(_processLogs) (1106) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:07,992::__init__::225::IOProcess::(_processLogs) DEBUG|(1106) Building response >Thread-17::DEBUG::2014-08-22 09:41:07,993::__init__::225::IOProcess::(_processLogs) (1106) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:07,993::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:07,993::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:41:08,161::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:41:08,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00aab2e-2fd0-4e88-8e21-2c73c0f6cf98`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:08,162::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:41:08,162::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:08,162::task::1191::Storage.TaskManager.Task::(prepare) Task=`f00aab2e-2fd0-4e88-8e21-2c73c0f6cf98`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:08,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00aab2e-2fd0-4e88-8e21-2c73c0f6cf98`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:08,162::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:08,162::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:08,162::task::993::Storage.TaskManager.Task::(_decref) Task=`f00aab2e-2fd0-4e88-8e21-2c73c0f6cf98`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:41:08,196::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:41:08,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`741ec8bb-e47d-40b5-8f02-dce43158bc17`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:41:08,196::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:41:08,196::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6eacb82f-9198-43de-adf0-864b2416d87a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:41:08,197::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:41:08,197::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:41:08,197::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6eacb82f-9198-43de-adf0-864b2416d87a`::Granted request >Thread-46::DEBUG::2014-08-22 09:41:08,197::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`741ec8bb-e47d-40b5-8f02-dce43158bc17`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:41:08,197::task::993::Storage.TaskManager.Task::(_decref) Task=`741ec8bb-e47d-40b5-8f02-dce43158bc17`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:41:08,199::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:08,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`741ec8bb-e47d-40b5-8f02-dce43158bc17`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:08,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`741ec8bb-e47d-40b5-8f02-dce43158bc17`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:41:08,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:41:08,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:41:08,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:41:08,200::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:41:08,200::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:41:08,200::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:41:08,200::task::993::Storage.TaskManager.Task::(_decref) Task=`741ec8bb-e47d-40b5-8f02-dce43158bc17`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) (1107) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) (1107) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) (1107) Building response >Thread-17::DEBUG::2014-08-22 09:41:08,755::__init__::225::IOProcess::(_processLogs) (1107) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:08,756::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:41:08,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:08,757::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:41:08,763::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000237664 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:08,763::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) (1108) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:08,764::__init__::225::IOProcess::(_processLogs) (1108) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:08,765::__init__::225::IOProcess::(_processLogs) DEBUG|(1108) Building response >Thread-17::DEBUG::2014-08-22 09:41:08,765::__init__::225::IOProcess::(_processLogs) (1108) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:08,766::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:08,766::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:08,766::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:08,766::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:41:08,766::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:08,766::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) (1109) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) (1109) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) (1109) Building response >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) (1109) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:08,767::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:08,768::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:08,768::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:41:08,768::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:08,768::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:08,768::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) (1110) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) (1110) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) (1110) Building response >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) (1110) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:08,769::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:41:08,770::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:08,770::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:08,770::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:08,770::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) (1111) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) (1111) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) (1111) Building response >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) (1111) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:08,771::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:41:09,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:09,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119403 s, 85.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:11,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:11,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116065 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:13,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:13,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116566 s, 87.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:41:15,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`97b3fcbf-8267-4ece-82ed-ba0f03050198`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:15,381::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:41:15,381::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000212196', 'lastCheck': '7.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237664', 'lastCheck': '6.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:15,381::task::1191::Storage.TaskManager.Task::(prepare) Task=`97b3fcbf-8267-4ece-82ed-ba0f03050198`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000212196', 'lastCheck': '7.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000237664', 'lastCheck': '6.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:15,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`97b3fcbf-8267-4ece-82ed-ba0f03050198`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:15,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:15,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:15,382::task::993::Storage.TaskManager.Task::(_decref) Task=`97b3fcbf-8267-4ece-82ed-ba0f03050198`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:41:15,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:15,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116082 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:17,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:17,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116777 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) (1112) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:17,994::__init__::225::IOProcess::(_processLogs) (1112) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:17,995::__init__::225::IOProcess::(_processLogs) DEBUG|(1112) Building response >Thread-21::DEBUG::2014-08-22 09:41:17,996::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:17,996::__init__::225::IOProcess::(_processLogs) (1112) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,002::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:41:18,003::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000159163 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:18,003::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:18,004::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:18,004::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:18,004::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:18,004::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:18,004::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:18,005::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:18,005::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:18,005::__init__::225::IOProcess::(_processLogs) (1113) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:18,005::__init__::225::IOProcess::(_processLogs) (1113) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:18,005::__init__::225::IOProcess::(_processLogs) DEBUG|(1113) Building response >Thread-17::DEBUG::2014-08-22 09:41:18,006::__init__::225::IOProcess::(_processLogs) (1113) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,006::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:18,006::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:41:18,249::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:41:18,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1efc675-9b94-4d52-82d1-f0ebf5eda6b6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:18,250::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:41:18,250::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:18,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1efc675-9b94-4d52-82d1-f0ebf5eda6b6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:18,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1efc675-9b94-4d52-82d1-f0ebf5eda6b6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:18,250::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:18,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:18,250::task::993::Storage.TaskManager.Task::(_decref) Task=`a1efc675-9b94-4d52-82d1-f0ebf5eda6b6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:41:18,274::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:41:18,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`244803c0-8141-4708-9047-1dae42d3f85c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:41:18,275::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:41:18,275::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e45e0e12-d029-4bf9-9e94-a12b6d81bc16`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:41:18,275::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:41:18,275::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:41:18,275::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e45e0e12-d029-4bf9-9e94-a12b6d81bc16`::Granted request >Thread-46::DEBUG::2014-08-22 09:41:18,276::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`244803c0-8141-4708-9047-1dae42d3f85c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:41:18,276::task::993::Storage.TaskManager.Task::(_decref) Task=`244803c0-8141-4708-9047-1dae42d3f85c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:41:18,278::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:18,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`244803c0-8141-4708-9047-1dae42d3f85c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:18,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`244803c0-8141-4708-9047-1dae42d3f85c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:41:18,278::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:41:18,278::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:41:18,278::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:41:18,278::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:41:18,278::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:41:18,278::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:41:18,278::task::993::Storage.TaskManager.Task::(_decref) Task=`244803c0-8141-4708-9047-1dae42d3f85c`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:41:18,771::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:18,771::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:18,771::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:18,771::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:18,771::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:18,771::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:18,772::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:18,772::__init__::225::IOProcess::(_processLogs) (1114) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:18,772::__init__::225::IOProcess::(_processLogs) (1114) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:18,772::__init__::225::IOProcess::(_processLogs) DEBUG|(1114) Building response >Thread-22::DEBUG::2014-08-22 09:41:18,773::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:18,773::__init__::225::IOProcess::(_processLogs) (1114) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:41:18,780::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000164154 s, 3.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:18,779::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:18,781::__init__::225::IOProcess::(_processLogs) (1115) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:18,782::__init__::225::IOProcess::(_processLogs) (1115) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:18,782::__init__::225::IOProcess::(_processLogs) DEBUG|(1115) Building response >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) (1115) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:18,783::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) (1116) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) (1116) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) (1116) Building response >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) (1116) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,784::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:18,785::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:18,785::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:41:18,785::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:18,785::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) (1117) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) (1117) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) (1117) Building response >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) (1117) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:18,786::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:41:18,787::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:18,787::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:18,787::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) (1118) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) (1118) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) (1118) Building response >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) (1118) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:18,788::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:41:19,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:19,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011916 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:21,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:22,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117266 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:24,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:24,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117676 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:26,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:26,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120651 s, 84.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) (1119) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:28,007::__init__::225::IOProcess::(_processLogs) (1119) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:28,008::__init__::225::IOProcess::(_processLogs) (1119) Queuing response >Thread-21::DEBUG::2014-08-22 09:41:28,009::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:28,009::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:28,015::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:41:28,016::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000205288 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:28,016::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:28,016::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:28,017::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,017::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,017::__init__::225::IOProcess::(_processLogs) (1120) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:28,017::__init__::225::IOProcess::(_processLogs) (1120) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:28,018::__init__::225::IOProcess::(_processLogs) (1120) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:28,018::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:28,018::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:41:28,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:28,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011549 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:41:28,329::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:41:28,329::task::595::Storage.TaskManager.Task::(_updateState) Task=`3350642d-8a7d-43ef-8462-cec0eabacba4`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:28,329::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:41:28,330::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:28,330::task::1191::Storage.TaskManager.Task::(prepare) Task=`3350642d-8a7d-43ef-8462-cec0eabacba4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:28,330::task::595::Storage.TaskManager.Task::(_updateState) Task=`3350642d-8a7d-43ef-8462-cec0eabacba4`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:28,330::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:28,330::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:28,330::task::993::Storage.TaskManager.Task::(_decref) Task=`3350642d-8a7d-43ef-8462-cec0eabacba4`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:41:28,373::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:41:28,373::task::595::Storage.TaskManager.Task::(_updateState) Task=`c954d701-3d5b-4131-9a58-0402ab1ec1c6`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:41:28,373::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:41:28,373::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8876021f-c513-4745-9aeb-0f72df8d3af2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:41:28,373::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:41:28,373::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:41:28,374::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8876021f-c513-4745-9aeb-0f72df8d3af2`::Granted request >Thread-46::DEBUG::2014-08-22 09:41:28,374::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c954d701-3d5b-4131-9a58-0402ab1ec1c6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:41:28,374::task::993::Storage.TaskManager.Task::(_decref) Task=`c954d701-3d5b-4131-9a58-0402ab1ec1c6`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:41:28,376::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:28,376::task::1191::Storage.TaskManager.Task::(prepare) Task=`c954d701-3d5b-4131-9a58-0402ab1ec1c6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:28,376::task::595::Storage.TaskManager.Task::(_updateState) Task=`c954d701-3d5b-4131-9a58-0402ab1ec1c6`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:41:28,376::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:41:28,376::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:41:28,376::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:41:28,376::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:41:28,377::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:41:28,377::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:41:28,377::task::993::Storage.TaskManager.Task::(_decref) Task=`c954d701-3d5b-4131-9a58-0402ab1ec1c6`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:41:28,788::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:28,788::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:28,788::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:28,788::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,788::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,789::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,789::__init__::225::IOProcess::(_processLogs) (1121) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:28,789::__init__::225::IOProcess::(_processLogs) (1121) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:28,789::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1121) Building response >Thread-22::DEBUG::2014-08-22 09:41:28,790::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:28,790::__init__::225::IOProcess::(_processLogs) (1121) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:28,796::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:41:28,797::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000264806 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:28,797::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) (1122) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:28,799::__init__::225::IOProcess::(_processLogs) (1122) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:28,800::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1122) Building response >Thread-17::DEBUG::2014-08-22 09:41:28,801::__init__::225::IOProcess::(_processLogs) (1122) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:28,801::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:28,801::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:28,801::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:28,801::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:41:28,801::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:28,802::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:28,802::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,802::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,802::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,802::__init__::225::IOProcess::(_processLogs) (1123) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) (1123) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) (1123) Building response >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) (1123) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:41:28,803::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) (1124) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) (1124) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:28,804::__init__::225::IOProcess::(_processLogs) (1124) Building response >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) (1124) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:28,805::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) (1125) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) (1125) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) (1125) Building response >Thread-17::DEBUG::2014-08-22 09:41:28,806::__init__::225::IOProcess::(_processLogs) (1125) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:28,807::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:28,807::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:41:30,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:30,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119214 s, 85.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:41:30,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb2babde-40a4-487b-b918-6093fbdf8ca0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:30,565::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:41:30,565::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205288', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264806', 'lastCheck': '1.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:30,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb2babde-40a4-487b-b918-6093fbdf8ca0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205288', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000264806', 'lastCheck': '1.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:30,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb2babde-40a4-487b-b918-6093fbdf8ca0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:30,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:30,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:30,566::task::993::Storage.TaskManager.Task::(_decref) Task=`bb2babde-40a4-487b-b918-6093fbdf8ca0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:41:32,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:32,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182802 s, 56.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:34,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:34,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116141 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:36,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:36,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116524 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:38,019::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,019::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:38,019::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,020::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:41:38,020::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:38,020::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,026::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:41:38,027::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227264 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:38,027::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,028::__init__::225::IOProcess::(_processLogs) (1126) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) (1126) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) (1126) Building response >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) (1126) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,029::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:38,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,030::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:38,030::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,030::__init__::225::IOProcess::(_processLogs) (1127) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:38,030::__init__::225::IOProcess::(_processLogs) (1127) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:38,030::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1127) Building response >Thread-17::DEBUG::2014-08-22 09:41:38,031::__init__::225::IOProcess::(_processLogs) (1127) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,031::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:38,031::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:41:38,432::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:41:38,432::task::595::Storage.TaskManager.Task::(_updateState) Task=`18d2f1ac-19fb-46a5-83da-fb0c8a82da7d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:38,432::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:41:38,432::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:38,432::task::1191::Storage.TaskManager.Task::(prepare) Task=`18d2f1ac-19fb-46a5-83da-fb0c8a82da7d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:38,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`18d2f1ac-19fb-46a5-83da-fb0c8a82da7d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:38,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:38,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:38,433::task::993::Storage.TaskManager.Task::(_decref) Task=`18d2f1ac-19fb-46a5-83da-fb0c8a82da7d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:41:38,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:38,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114404 s, 89.5 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:41:38,489::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:41:38,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`f62acaa1-8ca9-432a-a0de-86b9a1eaca11`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:41:38,491::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:41:38,492::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f32d9a12-738c-43c3-a6b6-154f1d056c77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:41:38,493::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:41:38,494::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:41:38,494::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f32d9a12-738c-43c3-a6b6-154f1d056c77`::Granted request >Thread-46::DEBUG::2014-08-22 09:41:38,496::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f62acaa1-8ca9-432a-a0de-86b9a1eaca11`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:41:38,497::task::993::Storage.TaskManager.Task::(_decref) Task=`f62acaa1-8ca9-432a-a0de-86b9a1eaca11`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:41:38,511::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:38,512::task::1191::Storage.TaskManager.Task::(prepare) Task=`f62acaa1-8ca9-432a-a0de-86b9a1eaca11`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:38,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`f62acaa1-8ca9-432a-a0de-86b9a1eaca11`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:41:38,512::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:41:38,513::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:41:38,513::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:41:38,514::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:41:38,514::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:41:38,514::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:41:38,515::task::993::Storage.TaskManager.Task::(_decref) Task=`f62acaa1-8ca9-432a-a0de-86b9a1eaca11`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:41:38,806::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) (1128) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:38,807::__init__::225::IOProcess::(_processLogs) (1128) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:38,808::__init__::225::IOProcess::(_processLogs) DEBUG|(1128) Building response >Thread-22::DEBUG::2014-08-22 09:41:38,809::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:38,809::__init__::225::IOProcess::(_processLogs) (1128) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,815::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:41:38,816::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00019834 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:38,816::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:38,817::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,817::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:38,817::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,817::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:38,817::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,817::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:38,818::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,818::__init__::225::IOProcess::(_processLogs) (1129) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:38,818::__init__::225::IOProcess::(_processLogs) (1129) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:38,818::__init__::225::IOProcess::(_processLogs) DEBUG|(1129) Building response >Thread-17::DEBUG::2014-08-22 09:41:38,819::__init__::225::IOProcess::(_processLogs) (1129) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,819::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:38,819::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:38,819::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,819::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:41:38,819::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) (1130) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) (1130) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:38,820::__init__::225::IOProcess::(_processLogs) (1130) Building response >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) (1130) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,821::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) (1131) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) (1131) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) (1131) Building response >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) (1131) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:38,822::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:38,823::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:38,823::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:41:38,823::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:38,823::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:38,823::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:38,825::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:38,825::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:38,825::__init__::225::IOProcess::(_processLogs) (1132) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:38,825::__init__::225::IOProcess::(_processLogs) (1132) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:38,826::__init__::225::IOProcess::(_processLogs) (1132) Building response >Thread-17::DEBUG::2014-08-22 09:41:38,826::__init__::225::IOProcess::(_processLogs) (1132) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:38,826::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:38,826::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:41:40,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:40,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118935 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:42,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:42,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116637 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:44,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:44,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113827 s, 90.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:41:45,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c95e504-961f-4515-b2ee-c0caf7b1d172`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:45,738::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:41:45,739::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227264', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00019834', 'lastCheck': '6.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:45,739::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c95e504-961f-4515-b2ee-c0caf7b1d172`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227264', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00019834', 'lastCheck': '6.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:41:45,739::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c95e504-961f-4515-b2ee-c0caf7b1d172`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:45,739::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:45,739::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:45,739::task::993::Storage.TaskManager.Task::(_decref) Task=`5c95e504-961f-4515-b2ee-c0caf7b1d172`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:41:46,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:46,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115942 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,032::__init__::225::IOProcess::(_processLogs) (1133) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:48,033::__init__::225::IOProcess::(_processLogs) (1133) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:48,033::__init__::225::IOProcess::(_processLogs) (1133) Queuing response >Thread-21::DEBUG::2014-08-22 09:41:48,034::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:48,034::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,040::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:41:48,041::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000202178 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) (1134) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:48,042::__init__::225::IOProcess::(_processLogs) (1134) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:48,043::__init__::225::IOProcess::(_processLogs) (1134) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:48,044::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,044::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:41:48,568::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-47::DEBUG::2014-08-22 09:41:48,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`af30027e-1b0d-48b0-81c7-32bf8f8ab906`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:48,568::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:41:48,568::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:48,568::task::1191::Storage.TaskManager.Task::(prepare) Task=`af30027e-1b0d-48b0-81c7-32bf8f8ab906`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:48,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`af30027e-1b0d-48b0-81c7-32bf8f8ab906`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:48,569::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:48,569::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:48,569::task::993::Storage.TaskManager.Task::(_decref) Task=`af30027e-1b0d-48b0-81c7-32bf8f8ab906`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:41:48,595::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-46::DEBUG::2014-08-22 09:41:48,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e3323bb-c8cd-41af-ba40-b8857dc049e8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:41:48,595::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:41:48,596::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`485fd6c9-2372-4561-a081-44ac1ec4b8b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:41:48,596::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:41:48,596::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:41:48,596::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`485fd6c9-2372-4561-a081-44ac1ec4b8b4`::Granted request >Thread-46::DEBUG::2014-08-22 09:41:48,596::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9e3323bb-c8cd-41af-ba40-b8857dc049e8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:41:48,596::task::993::Storage.TaskManager.Task::(_decref) Task=`9e3323bb-c8cd-41af-ba40-b8857dc049e8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:41:48,598::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:48,598::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e3323bb-c8cd-41af-ba40-b8857dc049e8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:48,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e3323bb-c8cd-41af-ba40-b8857dc049e8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:41:48,598::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:41:48,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:41:48,599::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:41:48,599::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:41:48,599::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:41:48,599::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:41:48,599::task::993::Storage.TaskManager.Task::(_decref) Task=`9e3323bb-c8cd-41af-ba40-b8857dc049e8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:41:48,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:48,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120297 s, 85.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:48,825::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:48,825::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:48,825::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,825::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,826::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,826::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,826::__init__::225::IOProcess::(_processLogs) (1135) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:48,826::__init__::225::IOProcess::(_processLogs) (1135) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:48,826::__init__::225::IOProcess::(_processLogs) DEBUG|(1135) Building response >Thread-17::DEBUG::2014-08-22 09:41:48,827::__init__::225::IOProcess::(_processLogs) (1135) Queuing response >Thread-22::DEBUG::2014-08-22 09:41:48,827::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:48,827::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,833::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:41:48,834::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218648 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:48,835::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:48,835::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:48,835::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:48,836::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) (1136) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) (1136) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) (1136) Building response >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) (1136) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,837::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:48,838::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:48,838::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:41:48,838::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:48,838::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) (1137) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) (1137) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) (1137) Building response >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) (1137) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:48,839::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:48,840::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:41:48,840::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:48,840::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,840::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) (1138) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) (1138) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) (1138) Building response >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) (1138) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:41:48,841::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:48,842::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:48,842::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) (1139) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) (1139) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) (1139) Building response >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) (1139) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:48,843::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:41:50,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:50,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117903 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:52,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:52,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118814 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:54,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:54,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117601 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:41:56,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:56,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114536 s, 89.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:58,044::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,044::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:58,044::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,044::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,045::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:58,045::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,045::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:58,045::__init__::225::IOProcess::(_processLogs) (1140) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:58,045::__init__::225::IOProcess::(_processLogs) (1140) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:58,045::__init__::225::IOProcess::(_processLogs) DEBUG|(1140) Building response >Thread-21::DEBUG::2014-08-22 09:41:58,046::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:58,046::__init__::225::IOProcess::(_processLogs) (1140) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,052::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:41:58,053::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219329 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:58,053::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:58,054::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) (1141) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:58,055::__init__::225::IOProcess::(_processLogs) (1141) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:58,056::__init__::225::IOProcess::(_processLogs) DEBUG|(1141) Building response >Thread-17::DEBUG::2014-08-22 09:41:58,057::__init__::225::IOProcess::(_processLogs) (1141) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,057::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:58,057::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:41:58,646::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:41:58,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`bcdae204-8c05-40c2-9605-869ea25d6a7c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:41:58,647::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:41:58,647::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:58,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`bcdae204-8c05-40c2-9605-869ea25d6a7c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:41:58,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`bcdae204-8c05-40c2-9605-869ea25d6a7c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:41:58,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:41:58,648::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:41:58,648::task::993::Storage.TaskManager.Task::(_decref) Task=`bcdae204-8c05-40c2-9605-869ea25d6a7c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:41:58,699::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:41:58,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`edfbdc71-fdc8-4001-a12f-82b39dc82a5e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:41:58,699::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:41:58,699::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`75272b5b-f720-43c7-83e1-e48224929929`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:41:58,699::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:41:58,699::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:41:58,699::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`75272b5b-f720-43c7-83e1-e48224929929`::Granted request >Thread-46::DEBUG::2014-08-22 09:41:58,700::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`edfbdc71-fdc8-4001-a12f-82b39dc82a5e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:41:58,700::task::993::Storage.TaskManager.Task::(_decref) Task=`edfbdc71-fdc8-4001-a12f-82b39dc82a5e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:41:58,702::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:58,702::task::1191::Storage.TaskManager.Task::(prepare) Task=`edfbdc71-fdc8-4001-a12f-82b39dc82a5e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:41:58,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`edfbdc71-fdc8-4001-a12f-82b39dc82a5e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:41:58,702::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:41:58,702::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:41:58,702::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:41:58,702::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:41:58,702::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:41:58,702::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:41:58,703::task::993::Storage.TaskManager.Task::(_decref) Task=`edfbdc71-fdc8-4001-a12f-82b39dc82a5e`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:41:58,843::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,843::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:58,843::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,844::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,845::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:41:58,845::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:41:58,845::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,851::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:41:58,852::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000203204 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:41:58,852::__init__::225::IOProcess::(_processLogs) (1142) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:58,853::__init__::225::IOProcess::(_processLogs) (1142) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:58,853::__init__::225::IOProcess::(_processLogs) (1142) Building response >Thread-17::DEBUG::2014-08-22 09:41:58,853::__init__::225::IOProcess::(_processLogs) (1142) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,853::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:58,853::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) (1143) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:41:58,854::__init__::225::IOProcess::(_processLogs) (1143) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:41:58,855::__init__::225::IOProcess::(_processLogs) DEBUG|(1143) Building response >Thread-17::DEBUG::2014-08-22 09:41:58,856::__init__::225::IOProcess::(_processLogs) (1143) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,856::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:58,856::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:41:58,856::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,856::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:41:58,856::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) (1144) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) (1144) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) (1144) Building response >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) (1144) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,857::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:58,858::__init__::225::IOProcess::(_processLogs) (1145) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:58,859::__init__::225::IOProcess::(_processLogs) (1145) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:58,859::__init__::225::IOProcess::(_processLogs) (1145) Building response >Thread-17::DEBUG::2014-08-22 09:41:58,859::__init__::225::IOProcess::(_processLogs) (1145) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,859::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:58,859::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:41:58,860::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:41:58,860::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:41:58,860::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:41:58,860::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:41:58,860::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) (1146) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) (1146) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) (1146) Building response >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) (1146) Queuing response >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:41:58,861::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:41:58,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:41:59,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114872 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:01,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:01,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122124 s, 83.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:42:01,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`bebaae19-3c8e-4399-8c74-e5ee3fd8332b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:01,207::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:42:01,208::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219329', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000203204', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:01,208::task::1191::Storage.TaskManager.Task::(prepare) Task=`bebaae19-3c8e-4399-8c74-e5ee3fd8332b`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219329', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000203204', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:01,208::task::595::Storage.TaskManager.Task::(_updateState) Task=`bebaae19-3c8e-4399-8c74-e5ee3fd8332b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:01,208::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:01,208::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:01,208::task::993::Storage.TaskManager.Task::(_decref) Task=`bebaae19-3c8e-4399-8c74-e5ee3fd8332b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:03,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:03,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119432 s, 85.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:05,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:05,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184208 s, 55.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:07,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:07,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012038 s, 85.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:08,058::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,059::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:08,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,060::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,061::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:08,061::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,064::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:08,065::__init__::225::IOProcess::(_processLogs) (1147) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:42:08,065::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:08,072::__init__::225::IOProcess::(_processLogs) (1147) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:08,073::__init__::225::IOProcess::(_processLogs) (1147) Building response >Thread-21::DEBUG::2014-08-22 09:42:08,074::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199056 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:08,074::__init__::225::IOProcess::(_processLogs) (1147) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,076::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,076::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:08,077::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) (1148) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:08,078::__init__::225::IOProcess::(_processLogs) (1148) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:08,079::__init__::225::IOProcess::(_processLogs) DEBUG|(1148) Building response >Thread-17::DEBUG::2014-08-22 09:42:08,079::__init__::225::IOProcess::(_processLogs) (1148) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,079::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,080::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:42:08,751::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:42:08,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`87d1779e-4fd2-41ad-bebe-4a6e673e9e67`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:08,752::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:42:08,752::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:08,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`87d1779e-4fd2-41ad-bebe-4a6e673e9e67`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:08,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`87d1779e-4fd2-41ad-bebe-4a6e673e9e67`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:08,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:08,752::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:08,752::task::993::Storage.TaskManager.Task::(_decref) Task=`87d1779e-4fd2-41ad-bebe-4a6e673e9e67`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:42:08,806::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:42:08,806::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c9ccda2-3c13-4f22-9474-30d1dc64fd84`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:42:08,806::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:42:08,807::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8497ad5a-6d31-49fd-920c-23cf7787452e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:42:08,807::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:42:08,807::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:42:08,807::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8497ad5a-6d31-49fd-920c-23cf7787452e`::Granted request >Thread-46::DEBUG::2014-08-22 09:42:08,807::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8c9ccda2-3c13-4f22-9474-30d1dc64fd84`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:42:08,807::task::993::Storage.TaskManager.Task::(_decref) Task=`8c9ccda2-3c13-4f22-9474-30d1dc64fd84`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:42:08,809::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:08,809::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c9ccda2-3c13-4f22-9474-30d1dc64fd84`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:08,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c9ccda2-3c13-4f22-9474-30d1dc64fd84`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:42:08,809::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:42:08,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:42:08,810::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:42:08,810::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:42:08,810::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:42:08,810::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:42:08,810::task::993::Storage.TaskManager.Task::(_decref) Task=`8c9ccda2-3c13-4f22-9474-30d1dc64fd84`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:42:08,861::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,861::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:08,861::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,862::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,863::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:42:08,863::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:08,863::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,869::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:42:08,870::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000314631 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:08,870::__init__::225::IOProcess::(_processLogs) (1149) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:08,871::__init__::225::IOProcess::(_processLogs) (1149) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:08,871::__init__::225::IOProcess::(_processLogs) (1149) Building response >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) (1149) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:08,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,873::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:08,873::__init__::225::IOProcess::(_processLogs) (1150) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:08,873::__init__::225::IOProcess::(_processLogs) (1150) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:08,873::__init__::225::IOProcess::(_processLogs) DEBUG|(1150) Building response >Thread-17::DEBUG::2014-08-22 09:42:08,874::__init__::225::IOProcess::(_processLogs) (1150) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,874::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,874::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:08,874::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,875::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:42:08,875::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,875::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,875::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:08,875::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) (1151) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) (1151) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) (1151) Building response >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) (1151) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:42:08,876::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,877::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,877::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:08,877::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,877::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:08,877::__init__::225::IOProcess::(_processLogs) (1152) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:08,877::__init__::225::IOProcess::(_processLogs) (1152) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) (1152) Building response >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) (1152) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:42:08,878::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:08,879::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:08,879::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:08,879::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:08,879::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:08,879::__init__::225::IOProcess::(_processLogs) (1153) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:08,879::__init__::225::IOProcess::(_processLogs) (1153) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:08,880::__init__::225::IOProcess::(_processLogs) (1153) Building response >Thread-17::DEBUG::2014-08-22 09:42:08,880::__init__::225::IOProcess::(_processLogs) (1153) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:08,880::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:08,880::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:42:09,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:09,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121132 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:11,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:11,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116636 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:13,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:13,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115852 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:15,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:15,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115924 s, 88.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:42:16,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cf53d01-cb13-4e09-85c2-f9bbed9bb826`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:16,441::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:42:16,441::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199056', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000314631', 'lastCheck': '7.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:16,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`5cf53d01-cb13-4e09-85c2-f9bbed9bb826`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199056', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000314631', 'lastCheck': '7.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:16,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cf53d01-cb13-4e09-85c2-f9bbed9bb826`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:16,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:16,441::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:16,442::task::993::Storage.TaskManager.Task::(_decref) Task=`5cf53d01-cb13-4e09-85c2-f9bbed9bb826`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:17,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:17,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122204 s, 83.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:18,080::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,080::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:18,080::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,080::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,080::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,080::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,081::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,081::__init__::225::IOProcess::(_processLogs) (1154) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:18,081::__init__::225::IOProcess::(_processLogs) (1154) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:18,081::__init__::225::IOProcess::(_processLogs) DEBUG|(1154) Building response >Thread-21::DEBUG::2014-08-22 09:42:18,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:18,082::__init__::225::IOProcess::(_processLogs) (1154) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:18,088::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,090::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:42:18,090::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000183974 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:18,093::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,093::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:18,094::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,095::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,095::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,096::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,096::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,097::__init__::225::IOProcess::(_processLogs) (1155) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:18,099::__init__::225::IOProcess::(_processLogs) DEBUG|(1155) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:18,100::__init__::225::IOProcess::(_processLogs) (1155) Building response >Thread-17::DEBUG::2014-08-22 09:42:18,101::__init__::225::IOProcess::(_processLogs) (1155) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:18,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,102::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:42:18,864::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:42:18,864::task::595::Storage.TaskManager.Task::(_updateState) Task=`362ea656-97d2-404e-94ab-2d9bfb1103ca`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:18,865::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:42:18,865::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:18,865::task::1191::Storage.TaskManager.Task::(prepare) Task=`362ea656-97d2-404e-94ab-2d9bfb1103ca`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:18,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`362ea656-97d2-404e-94ab-2d9bfb1103ca`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:18,865::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:18,865::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:18,865::task::993::Storage.TaskManager.Task::(_decref) Task=`362ea656-97d2-404e-94ab-2d9bfb1103ca`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:42:18,879::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) (1156) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:18,880::__init__::225::IOProcess::(_processLogs) (1156) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:18,881::__init__::225::IOProcess::(_processLogs) DEBUG|(1156) Building response >Thread-17::DEBUG::2014-08-22 09:42:18,881::__init__::225::IOProcess::(_processLogs) (1156) Queuing response >Thread-22::DEBUG::2014-08-22 09:42:18,881::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:18,882::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,887::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:42:18,888::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000193702 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,889::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,890::__init__::225::IOProcess::(_processLogs) (1157) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:18,890::__init__::225::IOProcess::(_processLogs) (1157) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:18,890::__init__::225::IOProcess::(_processLogs) DEBUG|(1157) Building response >Thread-17::DEBUG::2014-08-22 09:42:18,891::__init__::225::IOProcess::(_processLogs) (1157) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:18,891::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,891::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:18,891::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,891::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:42:18,892::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,892::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,892::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,892::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,892::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,892::__init__::225::IOProcess::(_processLogs) (1158) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) (1158) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) (1158) Building response >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) (1158) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:42:18,893::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) (1159) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) (1159) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) (1159) Building response >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) (1159) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:18,894::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,895::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:18,895::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:18,895::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:42:18,895::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:18,895::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:18,896::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:18,896::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:18,896::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:18,896::__init__::225::IOProcess::(_processLogs) (1160) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:18,896::__init__::225::IOProcess::(_processLogs) (1160) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:18,896::__init__::225::IOProcess::(_processLogs) (1160) Building response >Thread-17::DEBUG::2014-08-22 09:42:18,897::__init__::225::IOProcess::(_processLogs) (1160) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:18,897::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:18,897::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-46::DEBUG::2014-08-22 09:42:18,898::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:42:18,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`c280fb5e-2c4a-4735-964d-ade86f65d35d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:42:18,898::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:42:18,898::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`61395a1a-92f3-4fc9-afdd-19145ffa0552`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:42:18,898::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:42:18,899::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:42:18,899::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`61395a1a-92f3-4fc9-afdd-19145ffa0552`::Granted request >Thread-46::DEBUG::2014-08-22 09:42:18,899::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c280fb5e-2c4a-4735-964d-ade86f65d35d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:42:18,899::task::993::Storage.TaskManager.Task::(_decref) Task=`c280fb5e-2c4a-4735-964d-ade86f65d35d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:42:18,901::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:18,901::task::1191::Storage.TaskManager.Task::(prepare) Task=`c280fb5e-2c4a-4735-964d-ade86f65d35d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:18,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`c280fb5e-2c4a-4735-964d-ade86f65d35d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:42:18,901::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:42:18,901::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:42:18,901::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:42:18,902::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:42:18,902::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:42:18,902::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:42:18,902::task::993::Storage.TaskManager.Task::(_decref) Task=`c280fb5e-2c4a-4735-964d-ade86f65d35d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:19,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:19,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114225 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:21,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:21,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116456 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:23,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:23,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114514 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:25,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:25,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117892 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:27,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:27,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116382 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:28,101::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,101::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:28,101::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,101::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,101::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,101::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) (1161) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) (1161) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) (1161) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) (1161) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,102::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:42:28,103::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:42:28,109::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00026621 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:28,109::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,109::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) (1162) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) (1162) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) (1162) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) (1162) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,110::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,111::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:28,896::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) (1163) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) (1163) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) (1163) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,897::__init__::225::IOProcess::(_processLogs) (1163) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,898::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,899::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:42:28,899::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:42:28,905::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000278901 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,906::__init__::225::IOProcess::(_processLogs) (1164) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:28,907::__init__::225::IOProcess::(_processLogs) (1164) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:28,907::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1164) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,908::__init__::225::IOProcess::(_processLogs) (1164) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,908::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,908::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:28,908::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,908::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:42:28,909::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,909::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,909::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,909::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,909::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) (1165) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) (1165) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) (1165) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) (1165) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:42:28,910::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,911::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,911::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,911::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,911::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,911::__init__::225::IOProcess::(_processLogs) (1166) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:28,911::__init__::225::IOProcess::(_processLogs) (1166) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) (1166) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) (1166) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:42:28,912::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:28,913::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:28,913::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:28,913::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:28,913::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:28,913::__init__::225::IOProcess::(_processLogs) (1167) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:28,914::__init__::225::IOProcess::(_processLogs) (1167) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:28,914::__init__::225::IOProcess::(_processLogs) (1167) Building response >Thread-17::DEBUG::2014-08-22 09:42:28,914::__init__::225::IOProcess::(_processLogs) (1167) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:28,914::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:28,914::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:42:28,950::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:42:28,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`0889c0a5-059f-4a53-9810-cef220672094`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:28,951::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:42:28,951::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:28,951::task::1191::Storage.TaskManager.Task::(prepare) Task=`0889c0a5-059f-4a53-9810-cef220672094`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:28,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`0889c0a5-059f-4a53-9810-cef220672094`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:28,951::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:28,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:28,951::task::993::Storage.TaskManager.Task::(_decref) Task=`0889c0a5-059f-4a53-9810-cef220672094`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:42:28,976::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:42:28,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`44884348-9d70-4398-b8c4-4e5b4ce8db82`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:42:28,976::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:42:28,977::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c875b9d7-6730-4eaa-8c24-abd067e03a33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:42:28,977::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:42:28,977::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:42:28,977::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c875b9d7-6730-4eaa-8c24-abd067e03a33`::Granted request >Thread-46::DEBUG::2014-08-22 09:42:28,977::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`44884348-9d70-4398-b8c4-4e5b4ce8db82`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:42:28,977::task::993::Storage.TaskManager.Task::(_decref) Task=`44884348-9d70-4398-b8c4-4e5b4ce8db82`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:42:28,979::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:28,979::task::1191::Storage.TaskManager.Task::(prepare) Task=`44884348-9d70-4398-b8c4-4e5b4ce8db82`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:28,979::task::595::Storage.TaskManager.Task::(_updateState) Task=`44884348-9d70-4398-b8c4-4e5b4ce8db82`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:42:28,980::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:42:28,980::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:42:28,980::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:42:28,980::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:42:28,980::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:42:28,980::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:42:28,980::task::993::Storage.TaskManager.Task::(_decref) Task=`44884348-9d70-4398-b8c4-4e5b4ce8db82`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:29,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:29,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122602 s, 83.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:42:31,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ef9021a-f9b9-4848-9c2d-122e176253f8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:31,618::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:42:31,618::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00026621', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000278901', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:31,619::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ef9021a-f9b9-4848-9c2d-122e176253f8`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00026621', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000278901', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:31,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ef9021a-f9b9-4848-9c2d-122e176253f8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:31,619::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:31,619::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:31,619::task::993::Storage.TaskManager.Task::(_decref) Task=`5ef9021a-f9b9-4848-9c2d-122e176253f8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:31,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:31,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115304 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:33,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:33,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116565 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:35,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:35,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115632 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:37,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:37,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116731 s, 87.7 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:42:38,112::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:42:38,114::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,114::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:42:38,114::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,114::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,115::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,116::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,117::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,119::__init__::225::IOProcess::(_processLogs) (1168) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:42:38,120::__init__::225::IOProcess::(_processLogs) (1168) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:42:38,121::__init__::225::IOProcess::(_processLogs) (1168) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,121::__init__::225::IOProcess::(_processLogs) (1168) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,122::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,123::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:42:38,124::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,126::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-21::DEBUG::2014-08-22 09:42:38,126::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) (1169) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:42:38,127::__init__::225::IOProcess::(_processLogs) (1169) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:42:38,128::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1169) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,128::__init__::225::IOProcess::(_processLogs) (1169) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,128::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,128::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 09:42:38,128::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,128::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) (1170) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:42:38,129::__init__::225::IOProcess::(_processLogs) (1170) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:42:38,151::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1170) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,151::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1170) Queuing response >Thread-21::DEBUG::2014-08-22 09:42:38,151::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:42:38,152::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,153::__init__::225::IOProcess::(_processLogs) (1171) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:42:38,154::__init__::225::IOProcess::(_processLogs) (1171) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:42:38,154::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1171) Building response >Thread-21::DEBUG::2014-08-22 09:42:38,154::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:42:38,155::__init__::225::IOProcess::(_processLogs) (1171) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,156::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,156::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 09:42:38,156::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,156::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 09:42:38,156::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,157::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,157::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,157::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,157::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,157::__init__::225::IOProcess::(_processLogs) (1172) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:38,157::__init__::225::IOProcess::(_processLogs) (1172) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:38,158::__init__::225::IOProcess::(_processLogs) (1172) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,159::__init__::225::IOProcess::(_processLogs) (1172) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,160::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:38,161::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,162::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 09:42:38,162::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,163::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,163::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,164::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:42:38,164::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:42:38,166::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::INFO::2014-08-22 09:42:38,166::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:42:38,167::__init__::225::IOProcess::(_processLogs) (1173) Finding callback 'glob'... >Thread-21::INFO::2014-08-22 09:42:38,167::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:42:38,167::__init__::225::IOProcess::(_processLogs) (1173) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:42:38,168::__init__::225::IOProcess::(_processLogs) (1173) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,169::__init__::225::IOProcess::(_processLogs) (1173) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,169::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,170::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 09:42:38,171::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,171::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:38,173::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,174::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-21::DEBUG::2014-08-22 09:42:38,174::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:38,175::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,181::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,183::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:42:38,183::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000202279 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:38,184::__init__::225::IOProcess::(_processLogs) (1174) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:38,185::__init__::225::IOProcess::(_processLogs) (1174) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:38,186::__init__::225::IOProcess::(_processLogs) (1174) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,186::__init__::225::IOProcess::(_processLogs) (1174) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,187::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,187::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:38,188::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,188::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:38,189::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,191::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,192::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,193::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,193::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,194::__init__::225::IOProcess::(_processLogs) (1175) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:38,194::__init__::225::IOProcess::(_processLogs) (1175) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:38,195::__init__::225::IOProcess::(_processLogs) (1175) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,195::__init__::225::IOProcess::(_processLogs) (1175) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,196::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,196::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:42:38,913::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:42:38,915::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,915::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:42:38,915::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,916::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,916::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,916::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,916::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,916::__init__::225::IOProcess::(_processLogs) (1176) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:42:38,916::__init__::225::IOProcess::(_processLogs) (1176) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) (1176) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) (1176) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,917::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,918::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,919::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,920::__init__::225::IOProcess::(_processLogs) (1177) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:42:38,921::__init__::225::IOProcess::(_processLogs) (1177) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:42:38,921::__init__::225::IOProcess::(_processLogs) (1177) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,922::__init__::225::IOProcess::(_processLogs) (1177) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,923::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,923::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 09:42:38,924::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:42:38,924::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,924::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) (1178) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:42:38,925::__init__::225::IOProcess::(_processLogs) (1178) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:42:38,951::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1178) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,951::__init__::225::IOProcess::(_processLogs) (1178) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,951::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:42:38,951::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:42:38,952::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,953::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,954::__init__::225::IOProcess::(_processLogs) (1179) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:42:38,954::__init__::225::IOProcess::(_processLogs) (1179) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:42:38,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1179) Building response >Thread-22::DEBUG::2014-08-22 09:42:38,954::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:42:38,955::__init__::225::IOProcess::(_processLogs) (1179) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,956::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,956::__init__::225::IOProcess::(_processLogs) Sending response sized 832 >Thread-17::DEBUG::2014-08-22 09:42:38,956::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,957::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 09:42:38,957::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,957::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,957::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,957::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,957::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) (1180) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) (1180) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) (1180) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) (1180) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:42:38,958::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) (1181) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) (1181) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:42:38,959::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1181) Building response >Thread-22::DEBUG::2014-08-22 09:42:38,959::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:42:38,960::__init__::225::IOProcess::(_processLogs) (1181) Queuing response >Thread-22::INFO::2014-08-22 09:42:38,960::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:42:38,961::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 09:42:38,961::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:42:38,961::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 09:42:38,962::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,962::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:38,962::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,963::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:42:38,963::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:38,963::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,970::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:42:38,970::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199099 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:38,971::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,971::__init__::225::IOProcess::(_processLogs) (1182) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) (1182) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) (1182) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) (1182) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,972::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:38,973::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,973::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,973::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,973::__init__::225::IOProcess::(_processLogs) (1183) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:38,973::__init__::225::IOProcess::(_processLogs) (1183) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:38,973::__init__::225::IOProcess::(_processLogs) (1183) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,974::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,974::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:38,974::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:42:38,974::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,975::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,975::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,975::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,975::__init__::225::IOProcess::(_processLogs) (1184) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:38,975::__init__::225::IOProcess::(_processLogs) (1184) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:38,975::__init__::225::IOProcess::(_processLogs) (1184) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,976::__init__::225::IOProcess::(_processLogs) (1184) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,976::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,976::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:38,976::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:42:38,976::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,976::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) (1185) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) (1185) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) (1185) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) (1185) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,977::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:38,978::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:42:38,978::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:38,978::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:38,978::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) (1186) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) (1186) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) (1186) Building response >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) (1186) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:38,979::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:42:39,039::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:42:39,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4c3dde1-a04f-4fe5-99ab-ed92afec3e23`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:39,039::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:42:39,040::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:39,040::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4c3dde1-a04f-4fe5-99ab-ed92afec3e23`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:39,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4c3dde1-a04f-4fe5-99ab-ed92afec3e23`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:39,040::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:39,040::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:39,040::task::993::Storage.TaskManager.Task::(_decref) Task=`a4c3dde1-a04f-4fe5-99ab-ed92afec3e23`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:42:39,097::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:42:39,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5a0fd1b-c5af-4bb7-92e0-c615a99f8c45`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:42:39,098::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:42:39,098::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`de5c5acb-c4b2-4fe4-b79d-3d07d7eb60ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:42:39,098::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:42:39,098::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:42:39,098::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`de5c5acb-c4b2-4fe4-b79d-3d07d7eb60ab`::Granted request >Thread-46::DEBUG::2014-08-22 09:42:39,099::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a5a0fd1b-c5af-4bb7-92e0-c615a99f8c45`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:42:39,099::task::993::Storage.TaskManager.Task::(_decref) Task=`a5a0fd1b-c5af-4bb7-92e0-c615a99f8c45`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:42:39,101::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:39,101::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5a0fd1b-c5af-4bb7-92e0-c615a99f8c45`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:39,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5a0fd1b-c5af-4bb7-92e0-c615a99f8c45`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:42:39,101::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:42:39,101::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:42:39,101::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:42:39,101::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:42:39,101::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:42:39,101::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:42:39,101::task::993::Storage.TaskManager.Task::(_decref) Task=`a5a0fd1b-c5af-4bb7-92e0-c615a99f8c45`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:39,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:40,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121606 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:42,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:42,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117335 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:44,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:44,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117044 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:46,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:46,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118071 s, 86.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:42:46,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b1e9a58-046d-4961-bc75-46917d84d8b8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:46,793::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:42:46,793::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000202279', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199099', 'lastCheck': '7.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:46,794::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b1e9a58-046d-4961-bc75-46917d84d8b8`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000202279', 'lastCheck': '8.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199099', 'lastCheck': '7.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:42:46,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b1e9a58-046d-4961-bc75-46917d84d8b8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:46,794::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:46,794::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:46,794::task::993::Storage.TaskManager.Task::(_decref) Task=`7b1e9a58-046d-4961-bc75-46917d84d8b8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:48,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:48,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011457 s, 89.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:48,215::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:48,215::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:48,217::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:48,218::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:48,219::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:48,220::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,221::__init__::225::IOProcess::(_processLogs) (1187) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:48,222::__init__::225::IOProcess::(_processLogs) (1187) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:48,225::__init__::225::IOProcess::(_processLogs) (1187) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,226::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:42:48,226::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:48,227::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:42:48,235::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000197712 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:48,238::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:48,239::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:48,239::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:48,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:48,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,246::__init__::225::IOProcess::(_processLogs) (1188) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:48,247::__init__::225::IOProcess::(_processLogs) (1188) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:48,248::__init__::225::IOProcess::(_processLogs) (1188) Building response >Thread-17::DEBUG::2014-08-22 09:42:48,248::__init__::225::IOProcess::(_processLogs) (1188) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,249::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:48,250::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:48,979::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:48,979::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:48,980::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:42:48,981::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:48,981::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:48,987::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,988::__init__::225::IOProcess::(_processLogs) (1189) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:42:48,988::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216459 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:48,989::__init__::225::IOProcess::(_processLogs) (1189) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:48,989::__init__::225::IOProcess::(_processLogs) (1189) Building response >Thread-17::DEBUG::2014-08-22 09:42:48,989::__init__::225::IOProcess::(_processLogs) (1189) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,989::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:48,989::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) (1190) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:48,990::__init__::225::IOProcess::(_processLogs) (1190) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:48,991::__init__::225::IOProcess::(_processLogs) (1190) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,991::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:48,992::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:48,992::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:42:48,992::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:48,992::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) (1191) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) (1191) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) (1191) Building response >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) (1191) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:42:48,993::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:48,994::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:48,994::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:48,994::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,994::__init__::225::IOProcess::(_processLogs) (1192) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:48,994::__init__::225::IOProcess::(_processLogs) (1192) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:48,994::__init__::225::IOProcess::(_processLogs) (1192) Building response >Thread-17::DEBUG::2014-08-22 09:42:48,995::__init__::225::IOProcess::(_processLogs) (1192) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,995::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:48,995::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:48,995::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:42:48,995::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:48,995::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) (1193) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) (1193) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) (1193) Building response >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) (1193) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:48,996::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:48,997::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:42:49,150::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:42:49,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb99c8b3-f76c-4577-9921-b5ebb7eac26f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:49,151::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:42:49,151::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:49,151::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb99c8b3-f76c-4577-9921-b5ebb7eac26f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:49,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb99c8b3-f76c-4577-9921-b5ebb7eac26f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:49,151::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:49,151::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:49,151::task::993::Storage.TaskManager.Task::(_decref) Task=`bb99c8b3-f76c-4577-9921-b5ebb7eac26f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:42:49,185::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:42:49,185::task::595::Storage.TaskManager.Task::(_updateState) Task=`28c2ac30-5360-4a63-9930-6530b6838b87`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:42:49,185::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:42:49,186::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`089b69a6-2764-40cf-8337-96a669b1af5b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:42:49,186::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:42:49,186::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:42:49,187::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`089b69a6-2764-40cf-8337-96a669b1af5b`::Granted request >Thread-46::DEBUG::2014-08-22 09:42:49,187::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`28c2ac30-5360-4a63-9930-6530b6838b87`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:42:49,187::task::993::Storage.TaskManager.Task::(_decref) Task=`28c2ac30-5360-4a63-9930-6530b6838b87`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:42:49,189::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:49,189::task::1191::Storage.TaskManager.Task::(prepare) Task=`28c2ac30-5360-4a63-9930-6530b6838b87`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:49,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`28c2ac30-5360-4a63-9930-6530b6838b87`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:42:49,189::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:42:49,189::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:42:49,189::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:42:49,189::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:42:49,189::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:42:49,189::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:42:49,190::task::993::Storage.TaskManager.Task::(_decref) Task=`28c2ac30-5360-4a63-9930-6530b6838b87`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:42:50,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:50,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121501 s, 84.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:52,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:52,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117829 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:54,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:54,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116743 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:42:56,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:56,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011599 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:58,246::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:58,246::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) (1194) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) (1194) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) (1194) Building response >Thread-17::DEBUG::2014-08-22 09:42:58,247::__init__::225::IOProcess::(_processLogs) (1194) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:58,248::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:42:58,248::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:42:58,256::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000203417 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) (1195) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:58,257::__init__::225::IOProcess::(_processLogs) (1195) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:58,258::__init__::225::IOProcess::(_processLogs) (1195) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:58,259::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:58,259::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:42:58,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:42:58,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154575 s, 66.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) (1196) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:58,997::__init__::225::IOProcess::(_processLogs) (1196) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:58,998::__init__::225::IOProcess::(_processLogs) (1196) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:58,998::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:42:58,998::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:42:58,999::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:42:59,005::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000201272 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:42:59,005::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:42:59,006::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:42:59,006::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:59,006::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:59,006::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:59,006::__init__::225::IOProcess::(_processLogs) (1197) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:42:59,006::__init__::225::IOProcess::(_processLogs) (1197) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:42:59,007::__init__::225::IOProcess::(_processLogs) (1197) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:59,007::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:59,007::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:42:59,008::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:42:59,008::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:59,008::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:59,008::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:59,008::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) (1198) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) (1198) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) (1198) Building response >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) (1198) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:59,009::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:59,010::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:59,010::__init__::225::IOProcess::(_processLogs) (1199) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:59,010::__init__::225::IOProcess::(_processLogs) (1199) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:59,010::__init__::225::IOProcess::(_processLogs) (1199) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:59,010::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) (1200) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:42:59,011::__init__::225::IOProcess::(_processLogs) (1200) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:42:59,012::__init__::225::IOProcess::(_processLogs) (1200) Queuing response >Thread-17::DEBUG::2014-08-22 09:42:59,012::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:42:59,012::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:42:59,239::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:42:59,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`a14baf1c-2e97-4d3c-8835-cc0ca8e733f5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:42:59,239::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:42:59,240::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:59,240::task::1191::Storage.TaskManager.Task::(prepare) Task=`a14baf1c-2e97-4d3c-8835-cc0ca8e733f5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:42:59,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`a14baf1c-2e97-4d3c-8835-cc0ca8e733f5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:42:59,240::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:42:59,240::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:42:59,240::task::993::Storage.TaskManager.Task::(_decref) Task=`a14baf1c-2e97-4d3c-8835-cc0ca8e733f5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:42:59,262::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:42:59,262::task::595::Storage.TaskManager.Task::(_updateState) Task=`c59ecf71-6371-40f5-b288-983b0fa94e65`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:42:59,262::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:42:59,263::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`706c0a98-52b6-4535-93a2-a4498b623a1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:42:59,263::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:42:59,263::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:42:59,263::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`706c0a98-52b6-4535-93a2-a4498b623a1b`::Granted request >Thread-46::DEBUG::2014-08-22 09:42:59,263::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c59ecf71-6371-40f5-b288-983b0fa94e65`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:42:59,263::task::993::Storage.TaskManager.Task::(_decref) Task=`c59ecf71-6371-40f5-b288-983b0fa94e65`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:42:59,265::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:59,265::task::1191::Storage.TaskManager.Task::(prepare) Task=`c59ecf71-6371-40f5-b288-983b0fa94e65`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:42:59,265::task::595::Storage.TaskManager.Task::(_updateState) Task=`c59ecf71-6371-40f5-b288-983b0fa94e65`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:42:59,265::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:42:59,266::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:42:59,266::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:42:59,266::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:42:59,266::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:42:59,266::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:42:59,266::task::993::Storage.TaskManager.Task::(_decref) Task=`c59ecf71-6371-40f5-b288-983b0fa94e65`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:00,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:00,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114054 s, 89.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:01,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`66fff840-19cf-48f9-a330-b57d7a2c4035`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:01,970::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:43:01,971::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000203417', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000201272', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:01,972::task::1191::Storage.TaskManager.Task::(prepare) Task=`66fff840-19cf-48f9-a330-b57d7a2c4035`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000203417', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000201272', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:01,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`66fff840-19cf-48f9-a330-b57d7a2c4035`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:01,972::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:01,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:01,973::task::993::Storage.TaskManager.Task::(_decref) Task=`66fff840-19cf-48f9-a330-b57d7a2c4035`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:02,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:02,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205586 s, 49.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:04,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:04,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115654 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:06,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:06,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115577 s, 88.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:08,259::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:08,259::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:08,259::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:08,259::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:08,260::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:08,260::__init__::225::IOProcess::(_processLogs) (1201) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:08,260::__init__::225::IOProcess::(_processLogs) (1201) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:08,260::__init__::225::IOProcess::(_processLogs) (1201) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:08,261::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:43:08,261::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:08,262::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:43:08,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000174835 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) (1202) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:08,269::__init__::225::IOProcess::(_processLogs) (1202) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:08,270::__init__::225::IOProcess::(_processLogs) (1202) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:08,271::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:08,271::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:43:08,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:08,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118755 s, 86.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) (1203) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:09,013::__init__::225::IOProcess::(_processLogs) (1203) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:09,014::__init__::225::IOProcess::(_processLogs) (1203) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:09,015::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:43:09,015::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:09,015::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:43:09,022::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214713 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:09,022::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:09,022::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:09,022::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:09,022::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:09,023::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:09,023::__init__::225::IOProcess::(_processLogs) (1204) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:09,023::__init__::225::IOProcess::(_processLogs) (1204) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:09,023::__init__::225::IOProcess::(_processLogs) (1204) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:09,024::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:09,024::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:09,024::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:43:09,024::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:09,025::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:09,025::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:09,025::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:09,025::__init__::225::IOProcess::(_processLogs) (1205) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:09,025::__init__::225::IOProcess::(_processLogs) (1205) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:09,025::__init__::225::IOProcess::(_processLogs) (1205) Building response >Thread-17::DEBUG::2014-08-22 09:43:09,026::__init__::225::IOProcess::(_processLogs) (1205) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:09,026::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:09,026::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:09,026::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:43:09,026::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:09,026::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) (1206) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) (1206) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) (1206) Building response >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) (1206) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:09,027::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:43:09,028::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:09,028::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:09,028::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) (1207) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) (1207) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) (1207) Building response >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) (1207) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:09,029::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:43:09,317::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:43:09,317::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a2af15a-3878-44ff-b46e-3a1c865b7c7b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:09,317::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:43:09,318::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:09,318::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a2af15a-3878-44ff-b46e-3a1c865b7c7b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:09,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a2af15a-3878-44ff-b46e-3a1c865b7c7b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:09,318::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:09,318::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:09,318::task::993::Storage.TaskManager.Task::(_decref) Task=`5a2af15a-3878-44ff-b46e-3a1c865b7c7b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:43:09,380::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:43:09,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a575528-5ccc-4c41-b676-570dcbcca78f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:43:09,381::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:43:09,381::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`29b6167c-d49b-420d-a8c3-0f9357b47a56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:43:09,381::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:43:09,381::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:43:09,381::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`29b6167c-d49b-420d-a8c3-0f9357b47a56`::Granted request >Thread-46::DEBUG::2014-08-22 09:43:09,382::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8a575528-5ccc-4c41-b676-570dcbcca78f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:43:09,382::task::993::Storage.TaskManager.Task::(_decref) Task=`8a575528-5ccc-4c41-b676-570dcbcca78f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:43:09,384::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:09,384::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a575528-5ccc-4c41-b676-570dcbcca78f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:09,384::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a575528-5ccc-4c41-b676-570dcbcca78f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:43:09,384::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:43:09,384::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:43:09,384::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:43:09,384::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:43:09,384::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:43:09,384::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:43:09,384::task::993::Storage.TaskManager.Task::(_decref) Task=`8a575528-5ccc-4c41-b676-570dcbcca78f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:10,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:10,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117773 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:12,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:12,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115716 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:15,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:15,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116034 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:17,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:17,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116755 s, 87.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:17,159::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb9de09d-2998-4eb4-881a-83ceb53708b0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:17,159::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:43:17,160::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000174835', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214713', 'lastCheck': '8.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:17,160::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb9de09d-2998-4eb4-881a-83ceb53708b0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000174835', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214713', 'lastCheck': '8.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:17,160::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb9de09d-2998-4eb4-881a-83ceb53708b0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:17,160::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:17,160::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:17,160::task::993::Storage.TaskManager.Task::(_decref) Task=`fb9de09d-2998-4eb4-881a-83ceb53708b0`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:43:18,271::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:18,271::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:18,271::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:18,271::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:18,272::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:18,272::__init__::225::IOProcess::(_processLogs) (1208) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:18,272::__init__::225::IOProcess::(_processLogs) (1208) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:18,272::__init__::225::IOProcess::(_processLogs) (1208) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:18,273::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:43:18,273::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:18,274::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:43:18,280::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000197888 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:18,280::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) (1209) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:18,281::__init__::225::IOProcess::(_processLogs) (1209) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:18,282::__init__::225::IOProcess::(_processLogs) (1209) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:18,283::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:18,283::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:19,029::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:19,029::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:19,029::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:19,029::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:19,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:19,029::__init__::225::IOProcess::(_processLogs) (1210) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:19,030::__init__::225::IOProcess::(_processLogs) (1210) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:19,030::__init__::225::IOProcess::(_processLogs) (1210) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:19,030::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:43:19,031::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:19,031::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:43:19,037::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000174203 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) (1211) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:19,038::__init__::225::IOProcess::(_processLogs) (1211) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:19,039::__init__::225::IOProcess::(_processLogs) (1211) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:19,040::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:19,040::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:19,040::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:43:19,040::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:19,040::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) (1212) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) (1212) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) (1212) Building response >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) (1212) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:19,041::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:43:19,042::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:19,042::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:19,042::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:19,042::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:19,042::__init__::225::IOProcess::(_processLogs) (1213) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:19,042::__init__::225::IOProcess::(_processLogs) (1213) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) (1213) Building response >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) (1213) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:19,043::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:19,044::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:19,044::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:19,044::__init__::225::IOProcess::(_processLogs) (1214) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:19,044::__init__::225::IOProcess::(_processLogs) (1214) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:19,045::__init__::225::IOProcess::(_processLogs) (1214) Building response >Thread-17::DEBUG::2014-08-22 09:43:19,045::__init__::225::IOProcess::(_processLogs) (1214) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:19,045::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:19,045::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:43:19,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:19,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116337 s, 88.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:19,428::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:43:19,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`31294809-4c76-4337-97dc-c3500c479c33`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:19,429::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:43:19,429::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:19,429::task::1191::Storage.TaskManager.Task::(prepare) Task=`31294809-4c76-4337-97dc-c3500c479c33`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:19,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`31294809-4c76-4337-97dc-c3500c479c33`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:19,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:19,429::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:19,429::task::993::Storage.TaskManager.Task::(_decref) Task=`31294809-4c76-4337-97dc-c3500c479c33`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:43:19,463::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:43:19,463::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3cbac4c-62d1-43a8-b4c6-5db96a2f3d6b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:43:19,463::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:43:19,463::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a8c113d8-108d-4138-af8a-c20b407101c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:43:19,463::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:43:19,464::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:43:19,464::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a8c113d8-108d-4138-af8a-c20b407101c3`::Granted request >Thread-46::DEBUG::2014-08-22 09:43:19,464::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e3cbac4c-62d1-43a8-b4c6-5db96a2f3d6b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:43:19,464::task::993::Storage.TaskManager.Task::(_decref) Task=`e3cbac4c-62d1-43a8-b4c6-5db96a2f3d6b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:43:19,466::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:19,466::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3cbac4c-62d1-43a8-b4c6-5db96a2f3d6b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:19,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3cbac4c-62d1-43a8-b4c6-5db96a2f3d6b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:43:19,466::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:43:19,466::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:43:19,467::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:43:19,467::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:43:19,467::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:43:19,467::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:43:19,467::task::993::Storage.TaskManager.Task::(_decref) Task=`e3cbac4c-62d1-43a8-b4c6-5db96a2f3d6b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:21,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:21,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114413 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:23,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:23,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115709 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:25,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:25,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01169 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:27,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:27,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116117 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:28,283::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:28,283::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:28,284::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:28,284::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:28,284::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:28,284::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:28,284::__init__::225::IOProcess::(_processLogs) (1215) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:28,284::__init__::225::IOProcess::(_processLogs) (1215) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:28,285::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1215) Building response >Thread-21::DEBUG::2014-08-22 09:43:28,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:28,286::__init__::225::IOProcess::(_processLogs) (1215) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:28,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:43:28,293::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000158611 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:28,293::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:28,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:28,294::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:28,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:28,294::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:28,295::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:28,295::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:28,295::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:28,295::__init__::225::IOProcess::(_processLogs) (1216) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:28,295::__init__::225::IOProcess::(_processLogs) (1216) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:28,296::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1216) Building response >Thread-17::DEBUG::2014-08-22 09:43:28,296::__init__::225::IOProcess::(_processLogs) (1216) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:28,296::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:28,297::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:29,045::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:29,045::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:29,045::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:29,045::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:29,046::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:43:29,046::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:29,047::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:29,053::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:43:29,054::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000196588 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:29,054::__init__::225::IOProcess::(_processLogs) (1217) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) (1217) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) (1217) Building response >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) (1217) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:29,055::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) (1218) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:29,056::__init__::225::IOProcess::(_processLogs) (1218) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:29,057::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1218) Building response >Thread-17::DEBUG::2014-08-22 09:43:29,057::__init__::225::IOProcess::(_processLogs) (1218) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:29,057::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:29,058::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:29,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:29,058::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:43:29,058::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:29,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) (1219) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) (1219) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) (1219) Building response >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) (1219) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:29,059::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:43:29,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:29,060::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:29,060::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:29,060::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:29,060::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:29,060::__init__::225::IOProcess::(_processLogs) (1220) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) (1220) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) (1220) Building response >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) (1220) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:43:29,061::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:29,062::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:29,062::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:29,062::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:29,062::__init__::225::IOProcess::(_processLogs) (1221) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:29,062::__init__::225::IOProcess::(_processLogs) (1221) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:29,063::__init__::225::IOProcess::(_processLogs) (1221) Building response >Thread-17::DEBUG::2014-08-22 09:43:29,063::__init__::225::IOProcess::(_processLogs) (1221) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:29,063::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:29,063::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:43:29,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:29,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120585 s, 84.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:29,516::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:43:29,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`efefcb5c-144a-4f7c-af24-e61dee050165`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:29,516::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:43:29,516::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:29,517::task::1191::Storage.TaskManager.Task::(prepare) Task=`efefcb5c-144a-4f7c-af24-e61dee050165`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:29,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`efefcb5c-144a-4f7c-af24-e61dee050165`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:29,517::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:29,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:29,517::task::993::Storage.TaskManager.Task::(_decref) Task=`efefcb5c-144a-4f7c-af24-e61dee050165`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:43:29,576::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:43:29,576::task::595::Storage.TaskManager.Task::(_updateState) Task=`f487703d-dc5c-4b48-a833-b3c574ad6d17`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:43:29,576::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:43:29,576::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ed2dd908-a3c7-4c7e-a32b-bc6e84a7fc25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:43:29,576::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:43:29,576::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:43:29,577::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ed2dd908-a3c7-4c7e-a32b-bc6e84a7fc25`::Granted request >Thread-46::DEBUG::2014-08-22 09:43:29,577::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f487703d-dc5c-4b48-a833-b3c574ad6d17`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:43:29,577::task::993::Storage.TaskManager.Task::(_decref) Task=`f487703d-dc5c-4b48-a833-b3c574ad6d17`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:43:29,579::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:29,579::task::1191::Storage.TaskManager.Task::(prepare) Task=`f487703d-dc5c-4b48-a833-b3c574ad6d17`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:29,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`f487703d-dc5c-4b48-a833-b3c574ad6d17`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:43:29,579::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:43:29,579::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:43:29,579::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:43:29,579::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:43:29,580::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:43:29,580::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:43:29,580::task::993::Storage.TaskManager.Task::(_decref) Task=`f487703d-dc5c-4b48-a833-b3c574ad6d17`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:31,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:31,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118438 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:32,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`acae7c3c-87fe-4945-94c4-c0ab89e0d796`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:32,342::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:43:32,343::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000158611', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000196588', 'lastCheck': '3.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:32,343::task::1191::Storage.TaskManager.Task::(prepare) Task=`acae7c3c-87fe-4945-94c4-c0ab89e0d796`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000158611', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000196588', 'lastCheck': '3.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:32,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`acae7c3c-87fe-4945-94c4-c0ab89e0d796`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:32,344::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:32,344::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:32,345::task::993::Storage.TaskManager.Task::(_decref) Task=`acae7c3c-87fe-4945-94c4-c0ab89e0d796`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:33,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:33,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117402 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:35,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:35,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116491 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:37,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:37,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122442 s, 83.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:38,297::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:38,297::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:38,297::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:38,298::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:43:38,298::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:38,299::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:38,305::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:43:38,306::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000154577 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:38,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) (1222) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) (1222) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) (1222) Building response >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) (1222) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:38,307::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) (1223) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:38,308::__init__::225::IOProcess::(_processLogs) (1223) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:38,309::__init__::225::IOProcess::(_processLogs) (1223) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:38,310::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:38,310::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) (1224) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) (1224) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:39,063::__init__::225::IOProcess::(_processLogs) (1224) Building response >Thread-17::DEBUG::2014-08-22 09:43:39,064::__init__::225::IOProcess::(_processLogs) (1224) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:39,064::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:43:39,064::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:43:39,070::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00020856 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) (1225) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) (1225) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) (1225) Building response >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) (1225) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:39,071::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:39,072::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:43:39,072::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) (1226) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) (1226) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) (1226) Building response >Thread-17::DEBUG::2014-08-22 09:43:39,073::__init__::225::IOProcess::(_processLogs) (1226) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:39,074::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:39,074::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:39,074::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:43:39,074::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:39,074::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) (1227) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) (1227) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) (1227) Building response >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) (1227) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:39,075::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:43:39,076::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:39,076::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:39,076::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:39,077::__init__::225::IOProcess::(_processLogs) (1228) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:39,077::__init__::225::IOProcess::(_processLogs) (1228) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:39,077::__init__::225::IOProcess::(_processLogs) (1228) Building response >Thread-17::DEBUG::2014-08-22 09:43:39,077::__init__::225::IOProcess::(_processLogs) (1228) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:39,077::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:39,077::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:43:39,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:39,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121923 s, 84.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:39,828::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:43:39,828::task::595::Storage.TaskManager.Task::(_updateState) Task=`645c2b1b-3ffe-4bc3-a42d-5dffcdd50ecc`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:39,828::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:43:39,829::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:39,829::task::1191::Storage.TaskManager.Task::(prepare) Task=`645c2b1b-3ffe-4bc3-a42d-5dffcdd50ecc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:39,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`645c2b1b-3ffe-4bc3-a42d-5dffcdd50ecc`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:39,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:39,829::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:39,829::task::993::Storage.TaskManager.Task::(_decref) Task=`645c2b1b-3ffe-4bc3-a42d-5dffcdd50ecc`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:43:39,861::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:43:39,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`e92e0d72-1980-4f03-ba98-e0a5884acfde`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:43:39,862::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:43:39,862::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ae30e44f-bb92-4b57-ad15-fe7a68ee9d4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:43:39,862::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:43:39,862::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:43:39,862::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ae30e44f-bb92-4b57-ad15-fe7a68ee9d4b`::Granted request >Thread-46::DEBUG::2014-08-22 09:43:39,862::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e92e0d72-1980-4f03-ba98-e0a5884acfde`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:43:39,863::task::993::Storage.TaskManager.Task::(_decref) Task=`e92e0d72-1980-4f03-ba98-e0a5884acfde`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:43:39,865::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:39,865::task::1191::Storage.TaskManager.Task::(prepare) Task=`e92e0d72-1980-4f03-ba98-e0a5884acfde`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:39,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`e92e0d72-1980-4f03-ba98-e0a5884acfde`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:43:39,865::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:43:39,865::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:43:39,865::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:43:39,865::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:43:39,865::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:43:39,865::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:43:39,865::task::993::Storage.TaskManager.Task::(_decref) Task=`e92e0d72-1980-4f03-ba98-e0a5884acfde`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:41,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:41,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124493 s, 82.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:43,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:43,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115982 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:45,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:45,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116954 s, 87.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:43:47,606::task::595::Storage.TaskManager.Task::(_updateState) Task=`a81ed018-6585-4013-a201-51b5cb4b8665`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:47,606::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:43:47,606::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000154577', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020856', 'lastCheck': '8.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:47,607::task::1191::Storage.TaskManager.Task::(prepare) Task=`a81ed018-6585-4013-a201-51b5cb4b8665`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000154577', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020856', 'lastCheck': '8.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:43:47,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`a81ed018-6585-4013-a201-51b5cb4b8665`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:47,607::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:47,607::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:47,607::task::993::Storage.TaskManager.Task::(_decref) Task=`a81ed018-6585-4013-a201-51b5cb4b8665`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:47,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:47,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194798 s, 52.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:48,310::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:48,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:48,310::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:48,310::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:48,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:48,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:48,311::__init__::225::IOProcess::(_processLogs) (1229) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:48,311::__init__::225::IOProcess::(_processLogs) (1229) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:48,311::__init__::225::IOProcess::(_processLogs) DEBUG|(1229) Building response >Thread-17::DEBUG::2014-08-22 09:43:48,312::__init__::225::IOProcess::(_processLogs) (1229) Queuing response >Thread-21::DEBUG::2014-08-22 09:43:48,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:48,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:48,319::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:43:48,320::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000212142 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:48,320::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) (1230) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) (1230) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:48,321::__init__::225::IOProcess::(_processLogs) (1230) Building response >Thread-17::DEBUG::2014-08-22 09:43:48,322::__init__::225::IOProcess::(_processLogs) DEBUG|(1230) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:48,323::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:48,323::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:49,077::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:49,077::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:49,077::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:49,078::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:49,079::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:43:49,079::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:49,079::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:49,085::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:49,085::__init__::225::IOProcess::(_processLogs) (1231) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:49,085::__init__::225::IOProcess::(_processLogs) (1231) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:49,085::__init__::225::IOProcess::(_processLogs) (1231) Building response >Thread-17::DEBUG::2014-08-22 09:43:49,086::__init__::225::IOProcess::(_processLogs) (1231) Queuing response >Thread-22::DEBUG::2014-08-22 09:43:49,086::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00052663 s, 1.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:49,087::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:49,087::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) (1232) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:49,088::__init__::225::IOProcess::(_processLogs) (1232) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:49,104::__init__::225::IOProcess::(_processLogs) DEBUG|(1232) Building response >Thread-17::DEBUG::2014-08-22 09:43:49,104::__init__::225::IOProcess::(_processLogs) (1232) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:49,105::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:43:49,105::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:49,106::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:49,106::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:43:49,106::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:49,106::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) (1233) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) (1233) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) (1233) Building response >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) (1233) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:49,107::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:49,108::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:43:49,108::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:49,108::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:49,108::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) (1234) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) (1234) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) (1234) Building response >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) (1234) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:43:49,109::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:49,110::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:49,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:49,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) (1235) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) (1235) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) (1235) Building response >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) (1235) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:49,111::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:43:49,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-47::DEBUG::2014-08-22 09:43:49,914::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:43:49,915::task::595::Storage.TaskManager.Task::(_updateState) Task=`04e47800-fc1a-4160-a4f4-52f984f84ee2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:43:49,916::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:43:49,917::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:49,917::task::1191::Storage.TaskManager.Task::(prepare) Task=`04e47800-fc1a-4160-a4f4-52f984f84ee2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:43:49,918::task::595::Storage.TaskManager.Task::(_updateState) Task=`04e47800-fc1a-4160-a4f4-52f984f84ee2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:43:49,918::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:43:49,919::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:43:49,919::task::993::Storage.TaskManager.Task::(_decref) Task=`04e47800-fc1a-4160-a4f4-52f984f84ee2`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:49,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177172 s, 57.8 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:43:49,947::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:43:49,948::task::595::Storage.TaskManager.Task::(_updateState) Task=`b786f118-f8e4-46c5-98c8-bfbcb98f16f6`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:43:49,949::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:43:49,950::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b75019fb-ce16-4d85-af84-d8d687682659`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:43:49,950::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:43:49,951::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:43:49,951::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b75019fb-ce16-4d85-af84-d8d687682659`::Granted request >Thread-46::DEBUG::2014-08-22 09:43:49,952::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b786f118-f8e4-46c5-98c8-bfbcb98f16f6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:43:49,952::task::993::Storage.TaskManager.Task::(_decref) Task=`b786f118-f8e4-46c5-98c8-bfbcb98f16f6`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:43:49,961::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:49,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`b786f118-f8e4-46c5-98c8-bfbcb98f16f6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:43:49,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`b786f118-f8e4-46c5-98c8-bfbcb98f16f6`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:43:49,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:43:49,963::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:43:49,963::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:43:49,964::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:43:49,964::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:43:49,964::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:43:49,965::task::993::Storage.TaskManager.Task::(_decref) Task=`b786f118-f8e4-46c5-98c8-bfbcb98f16f6`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:43:51,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:52,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119705 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:54,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:54,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120552 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:56,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:56,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0126274 s, 81.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:43:58,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:43:58,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125166 s, 81.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:58,324::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:58,324::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:58,324::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:58,324::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:43:58,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:43:58,325::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:58,331::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:43:58,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000166478 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:58,332::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:58,333::__init__::225::IOProcess::(_processLogs) (1236) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) (1236) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) (1236) Building response >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) (1236) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:58,334::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) (1237) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) (1237) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) (1237) Building response >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) (1237) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:58,335::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) (1238) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:59,111::__init__::225::IOProcess::(_processLogs) (1238) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:59,114::__init__::225::IOProcess::(_processLogs) (1238) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:59,114::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:43:59,115::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:43:59,121::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000315765 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:43:59,121::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:43:59,121::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:59,122::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:59,122::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:59,122::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:59,122::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:59,122::__init__::225::IOProcess::(_processLogs) (1239) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:43:59,122::__init__::225::IOProcess::(_processLogs) (1239) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:43:59,123::__init__::225::IOProcess::(_processLogs) DEBUG|(1239) Building response >Thread-17::DEBUG::2014-08-22 09:43:59,123::__init__::225::IOProcess::(_processLogs) (1239) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:59,123::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:59,123::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:43:59,124::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:59,124::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:43:59,124::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:59,124::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:59,124::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) (1240) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) (1240) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) (1240) Building response >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) (1240) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:59,125::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:43:59,126::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:59,126::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:59,126::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:43:59,126::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:59,126::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) (1241) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) (1241) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) (1241) Building response >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) (1241) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:43:59,127::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:43:59,128::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:43:59,128::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) (1242) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) (1242) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) (1242) Building response >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) (1242) Queuing response >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:43:59,129::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:44:00,028::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:44:00,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8d6f855-1a56-4adc-91d8-630d14b79c1c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:00,028::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:44:00,029::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:00,029::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8d6f855-1a56-4adc-91d8-630d14b79c1c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:00,029::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8d6f855-1a56-4adc-91d8-630d14b79c1c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:00,029::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:00,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:00,029::task::993::Storage.TaskManager.Task::(_decref) Task=`e8d6f855-1a56-4adc-91d8-630d14b79c1c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:44:00,094::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:00,094::task::595::Storage.TaskManager.Task::(_updateState) Task=`21748c14-6547-4c43-aca3-25561c7c38d4`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:00,094::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:44:00,095::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`415c8acc-79c9-455b-b484-a9b4c31f0126`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:44:00,095::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:00,095::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:00,095::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`415c8acc-79c9-455b-b484-a9b4c31f0126`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:00,095::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`21748c14-6547-4c43-aca3-25561c7c38d4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:44:00,095::task::993::Storage.TaskManager.Task::(_decref) Task=`21748c14-6547-4c43-aca3-25561c7c38d4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:44:00,097::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:00,097::task::1191::Storage.TaskManager.Task::(prepare) Task=`21748c14-6547-4c43-aca3-25561c7c38d4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:00,097::task::595::Storage.TaskManager.Task::(_updateState) Task=`21748c14-6547-4c43-aca3-25561c7c38d4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:44:00,097::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:44:00,098::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:44:00,098::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:44:00,098::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:44:00,098::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:44:00,098::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:44:00,098::task::993::Storage.TaskManager.Task::(_decref) Task=`21748c14-6547-4c43-aca3-25561c7c38d4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:00,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:00,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013383 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:02,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:02,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212955 s, 48.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:44:03,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`4491134a-97f6-4730-bd0f-6635c8529589`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:03,015::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:44:03,015::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000166478', 'lastCheck': '4.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000315765', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:03,015::task::1191::Storage.TaskManager.Task::(prepare) Task=`4491134a-97f6-4730-bd0f-6635c8529589`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000166478', 'lastCheck': '4.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000315765', 'lastCheck': '3.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:03,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`4491134a-97f6-4730-bd0f-6635c8529589`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:03,015::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:03,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:03,016::task::993::Storage.TaskManager.Task::(_decref) Task=`4491134a-97f6-4730-bd0f-6635c8529589`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:04,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:04,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116819 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:06,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:06,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116471 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) (1243) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:08,337::__init__::225::IOProcess::(_processLogs) (1243) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:08,338::__init__::225::IOProcess::(_processLogs) DEBUG|(1243) Building response >Thread-17::DEBUG::2014-08-22 09:44:08,338::__init__::225::IOProcess::(_processLogs) (1243) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:08,338::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 09:44:08,339::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:08,339::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:44:08,346::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000313671 s, 1.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:08,346::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:08,346::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:08,346::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) (1244) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) (1244) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) (1244) Building response >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) (1244) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:08,347::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:08,348::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:44:08,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:08,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263209 s, 38.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:09,129::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:09,129::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:09,129::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:09,130::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:44:09,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:09,131::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:09,137::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:44:09,138::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199864 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:09,138::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) (1245) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) (1245) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) (1245) Building response >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) (1245) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:09,139::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) (1246) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:09,140::__init__::225::IOProcess::(_processLogs) (1246) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:09,141::__init__::225::IOProcess::(_processLogs) DEBUG|(1246) Building response >Thread-17::DEBUG::2014-08-22 09:44:09,141::__init__::225::IOProcess::(_processLogs) (1246) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:09,142::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:09,142::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:09,142::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:09,142::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:44:09,142::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) (1247) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) (1247) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:09,143::__init__::225::IOProcess::(_processLogs) (1247) Building response >Thread-17::DEBUG::2014-08-22 09:44:09,144::__init__::225::IOProcess::(_processLogs) (1247) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:09,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:09,144::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:09,144::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:09,144::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:44:09,144::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) (1248) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) (1248) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:09,145::__init__::225::IOProcess::(_processLogs) (1248) Building response >Thread-17::DEBUG::2014-08-22 09:44:09,146::__init__::225::IOProcess::(_processLogs) (1248) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:09,146::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:09,146::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:09,146::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:09,146::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:44:09,146::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) (1249) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) (1249) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:09,147::__init__::225::IOProcess::(_processLogs) (1249) Building response >Thread-17::DEBUG::2014-08-22 09:44:09,148::__init__::225::IOProcess::(_processLogs) (1249) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:09,148::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:09,148::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:44:10,224::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:44:10,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`423c11cb-d1b3-4b33-8f2b-0e999ab66354`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:10,224::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:44:10,225::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:10,225::task::1191::Storage.TaskManager.Task::(prepare) Task=`423c11cb-d1b3-4b33-8f2b-0e999ab66354`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:10,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`423c11cb-d1b3-4b33-8f2b-0e999ab66354`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:10,225::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:10,225::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:10,225::task::993::Storage.TaskManager.Task::(_decref) Task=`423c11cb-d1b3-4b33-8f2b-0e999ab66354`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:44:10,282::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:10,282::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c2460f3-7174-4241-8762-faf7f72ad8d4`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:10,282::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:44:10,282::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1a759f07-5f14-474e-8a10-27b0618d5e0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:44:10,282::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:10,283::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:10,283::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1a759f07-5f14-474e-8a10-27b0618d5e0f`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:10,283::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1c2460f3-7174-4241-8762-faf7f72ad8d4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:44:10,283::task::993::Storage.TaskManager.Task::(_decref) Task=`1c2460f3-7174-4241-8762-faf7f72ad8d4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:44:10,285::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:10,285::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c2460f3-7174-4241-8762-faf7f72ad8d4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:10,285::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c2460f3-7174-4241-8762-faf7f72ad8d4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:44:10,285::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:44:10,285::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:44:10,285::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:44:10,285::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:44:10,286::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:44:10,286::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:44:10,286::task::993::Storage.TaskManager.Task::(_decref) Task=`1c2460f3-7174-4241-8762-faf7f72ad8d4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:10,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:10,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117686 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:12,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:12,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119138 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:14,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:14,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114544 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:16,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:16,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118227 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:44:18,271::task::595::Storage.TaskManager.Task::(_updateState) Task=`a700d1be-751a-47b7-8087-e6f1c3423070`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:18,271::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:44:18,271::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000313671', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199864', 'lastCheck': '9.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:18,271::task::1191::Storage.TaskManager.Task::(prepare) Task=`a700d1be-751a-47b7-8087-e6f1c3423070`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000313671', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000199864', 'lastCheck': '9.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:18,271::task::595::Storage.TaskManager.Task::(_updateState) Task=`a700d1be-751a-47b7-8087-e6f1c3423070`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:18,271::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:18,271::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:18,272::task::993::Storage.TaskManager.Task::(_decref) Task=`a700d1be-751a-47b7-8087-e6f1c3423070`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:44:18,349::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:18,349::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:18,349::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:18,350::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:18,350::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:18,350::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:18,350::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:18,350::__init__::225::IOProcess::(_processLogs) (1250) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:18,350::__init__::225::IOProcess::(_processLogs) (1250) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:18,351::__init__::225::IOProcess::(_processLogs) DEBUG|(1250) Building response >Thread-21::DEBUG::2014-08-22 09:44:18,351::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:18,351::__init__::225::IOProcess::(_processLogs) (1250) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:18,357::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:18,358::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:44:18,359::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000152305 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) (1251) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:18,360::__init__::225::IOProcess::(_processLogs) (1251) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:18,361::__init__::225::IOProcess::(_processLogs) DEBUG|(1251) Building response >Thread-17::DEBUG::2014-08-22 09:44:18,362::__init__::225::IOProcess::(_processLogs) (1251) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:18,362::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:18,362::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:44:18,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:18,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117692 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) (1252) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:19,148::__init__::225::IOProcess::(_processLogs) (1252) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:19,149::__init__::225::IOProcess::(_processLogs) DEBUG|(1252) Building response >Thread-17::DEBUG::2014-08-22 09:44:19,149::__init__::225::IOProcess::(_processLogs) (1252) Queuing response >Thread-22::DEBUG::2014-08-22 09:44:19,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:19,150::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:19,155::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:44:19,156::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000153443 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:19,156::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:19,156::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) (1253) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) (1253) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) (1253) Building response >Thread-17::DEBUG::2014-08-22 09:44:19,157::__init__::225::IOProcess::(_processLogs) (1253) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:19,158::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:44:19,159::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:19,159::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:19,159::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:44:19,159::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:19,159::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) (1254) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) (1254) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) (1254) Building response >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) (1254) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:19,160::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:19,161::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:44:19,161::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:19,161::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:19,161::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:19,161::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) (1255) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) (1255) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) (1255) Building response >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) (1255) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:44:19,162::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:19,163::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:19,163::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:19,163::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) (1256) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) (1256) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) (1256) Building response >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) (1256) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:19,164::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:44:20,735::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:44:20,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7efd3a4-896f-48a6-8ee4-7140a4424c22`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:20,735::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:44:20,736::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:20,736::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7efd3a4-896f-48a6-8ee4-7140a4424c22`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:20,736::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7efd3a4-896f-48a6-8ee4-7140a4424c22`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:20,736::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:20,736::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:20,736::task::993::Storage.TaskManager.Task::(_decref) Task=`c7efd3a4-896f-48a6-8ee4-7140a4424c22`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:44:20,784::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:20,784::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ee86cca-6c7e-409b-baff-f36bef0ef693`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:20,784::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:44:20,784::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bfcd3d7b-b88e-4215-a58d-d67dd8ed1ef0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:44:20,784::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:20,785::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:20,785::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bfcd3d7b-b88e-4215-a58d-d67dd8ed1ef0`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:20,785::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4ee86cca-6c7e-409b-baff-f36bef0ef693`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:44:20,785::task::993::Storage.TaskManager.Task::(_decref) Task=`4ee86cca-6c7e-409b-baff-f36bef0ef693`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:44:20,787::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:20,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ee86cca-6c7e-409b-baff-f36bef0ef693`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:20,787::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ee86cca-6c7e-409b-baff-f36bef0ef693`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:44:20,787::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:44:20,787::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:44:20,787::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:44:20,787::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:44:20,787::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:44:20,788::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:44:20,788::task::993::Storage.TaskManager.Task::(_decref) Task=`4ee86cca-6c7e-409b-baff-f36bef0ef693`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:20,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:20,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120645 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:22,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:22,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114623 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:25,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:25,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012061 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:27,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:27,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114717 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:28,362::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) (1257) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:28,363::__init__::225::IOProcess::(_processLogs) (1257) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:28,364::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1257) Building response >Thread-17::DEBUG::2014-08-22 09:44:28,365::__init__::225::IOProcess::(_processLogs) (1257) Queuing response >Thread-21::DEBUG::2014-08-22 09:44:28,365::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:28,365::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:28,371::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:44:28,372::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000207811 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:28,373::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:28,374::__init__::225::IOProcess::(_processLogs) (1258) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:28,374::__init__::225::IOProcess::(_processLogs) (1258) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:28,374::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1258) Building response >Thread-17::DEBUG::2014-08-22 09:44:28,375::__init__::225::IOProcess::(_processLogs) (1258) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:28,375::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:28,375::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:44:29,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:29,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116682 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:29,164::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:29,164::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:29,164::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:29,164::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:29,165::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:29,165::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:29,165::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:29,165::__init__::225::IOProcess::(_processLogs) (1259) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:29,165::__init__::225::IOProcess::(_processLogs) (1259) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:29,165::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1259) Building response >Thread-17::DEBUG::2014-08-22 09:44:29,166::__init__::225::IOProcess::(_processLogs) (1259) Queuing response >Thread-22::DEBUG::2014-08-22 09:44:29,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:29,166::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:29,172::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:44:29,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214138 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:29,174::__init__::225::IOProcess::(_processLogs) (1260) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:29,175::__init__::225::IOProcess::(_processLogs) (1260) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:29,175::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1260) Building response >Thread-17::DEBUG::2014-08-22 09:44:29,176::__init__::225::IOProcess::(_processLogs) (1260) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:29,176::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:29,176::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:29,176::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:29,176::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:44:29,176::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) (1261) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) (1261) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) (1261) Building response >Thread-17::DEBUG::2014-08-22 09:44:29,177::__init__::225::IOProcess::(_processLogs) (1261) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:29,178::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:29,178::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:29,178::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:29,178::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:44:29,178::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:29,178::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) (1262) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) (1262) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) (1262) Building response >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) (1262) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:29,179::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:29,180::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:29,180::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:44:29,180::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:29,180::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:29,180::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) (1263) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) (1263) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) (1263) Building response >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) (1263) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:29,181::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:44:30,836::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:44:30,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`bbc9f129-dde4-425d-8ca9-9bac2237749c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:30,837::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:44:30,837::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:30,837::task::1191::Storage.TaskManager.Task::(prepare) Task=`bbc9f129-dde4-425d-8ca9-9bac2237749c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:30,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`bbc9f129-dde4-425d-8ca9-9bac2237749c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:30,837::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:30,837::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:30,837::task::993::Storage.TaskManager.Task::(_decref) Task=`bbc9f129-dde4-425d-8ca9-9bac2237749c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:44:30,871::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:30,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b712395-7da4-42f4-ba66-ebb5ee32c17d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:30,872::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:44:30,872::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`aa51d7d3-c64e-4c75-ab11-81557c01c9f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:44:30,872::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:30,872::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:30,872::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`aa51d7d3-c64e-4c75-ab11-81557c01c9f9`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:30,872::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6b712395-7da4-42f4-ba66-ebb5ee32c17d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:44:30,873::task::993::Storage.TaskManager.Task::(_decref) Task=`6b712395-7da4-42f4-ba66-ebb5ee32c17d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:44:30,874::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:30,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b712395-7da4-42f4-ba66-ebb5ee32c17d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:30,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b712395-7da4-42f4-ba66-ebb5ee32c17d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:44:30,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:44:30,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:44:30,875::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:44:30,875::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:44:30,875::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:44:30,875::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:44:30,875::task::993::Storage.TaskManager.Task::(_decref) Task=`6b712395-7da4-42f4-ba66-ebb5ee32c17d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:31,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:31,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011896 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:33,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:33,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116684 s, 87.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:44:33,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`047e8dc3-86ab-49c5-84a1-1b0f84ca5f38`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:33,517::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:44:33,517::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000207811', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214138', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:33,517::task::1191::Storage.TaskManager.Task::(prepare) Task=`047e8dc3-86ab-49c5-84a1-1b0f84ca5f38`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000207811', 'lastCheck': '5.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214138', 'lastCheck': '4.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:33,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`047e8dc3-86ab-49c5-84a1-1b0f84ca5f38`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:33,517::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:33,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:33,518::task::993::Storage.TaskManager.Task::(_decref) Task=`047e8dc3-86ab-49c5-84a1-1b0f84ca5f38`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:35,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:35,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115763 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:37,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:37,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117068 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) (1264) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:38,376::__init__::225::IOProcess::(_processLogs) (1264) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:38,377::__init__::225::IOProcess::(_processLogs) (1264) Building response >Thread-21::DEBUG::2014-08-22 09:44:38,378::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:38,378::__init__::225::IOProcess::(_processLogs) (1264) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:38,384::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:44:38,385::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000204742 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:38,385::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) (1265) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:38,386::__init__::225::IOProcess::(_processLogs) (1265) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:38,387::__init__::225::IOProcess::(_processLogs) (1265) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:38,388::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:38,388::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:39,181::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:39,181::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:39,181::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:39,182::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:39,182::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:39,182::__init__::225::IOProcess::(_processLogs) (1266) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:39,182::__init__::225::IOProcess::(_processLogs) (1266) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:39,182::__init__::225::IOProcess::(_processLogs) (1266) Queuing response >Thread-22::DEBUG::2014-08-22 09:44:39,183::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:39,183::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:39,189::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:44:39,191::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00139624 s, 408 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) (1267) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:39,192::__init__::225::IOProcess::(_processLogs) (1267) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:39,193::__init__::225::IOProcess::(_processLogs) (1267) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:39,194::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:39,195::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:44:39,195::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:39,195::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) (1268) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) (1268) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) (1268) Building response >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) (1268) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:44:39,196::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:39,197::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:39,197::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:39,197::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:39,197::__init__::225::IOProcess::(_processLogs) (1269) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) (1269) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) (1269) Building response >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) (1269) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:44:39,198::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:39,199::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:39,199::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:39,199::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:39,199::__init__::225::IOProcess::(_processLogs) (1270) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:39,200::__init__::225::IOProcess::(_processLogs) (1270) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:39,200::__init__::225::IOProcess::(_processLogs) (1270) Building response >Thread-17::DEBUG::2014-08-22 09:44:39,200::__init__::225::IOProcess::(_processLogs) (1270) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:39,200::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:39,200::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:44:39,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:39,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117176 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:44:40,956::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:44:40,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`efc80fe4-c7f2-4e0d-b1a7-0296931768ae`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:40,956::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:44:40,957::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:40,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`efc80fe4-c7f2-4e0d-b1a7-0296931768ae`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:40,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`efc80fe4-c7f2-4e0d-b1a7-0296931768ae`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:40,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:40,957::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:40,957::task::993::Storage.TaskManager.Task::(_decref) Task=`efc80fe4-c7f2-4e0d-b1a7-0296931768ae`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:44:41,021::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:41,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d1250b3-2f1d-4295-a340-b8cc3a791279`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:41,021::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:44:41,022::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ce568c66-4204-4000-b1bb-16b92c5fc5b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:44:41,022::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:41,022::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:41,022::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ce568c66-4204-4000-b1bb-16b92c5fc5b8`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:41,022::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8d1250b3-2f1d-4295-a340-b8cc3a791279`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:44:41,022::task::993::Storage.TaskManager.Task::(_decref) Task=`8d1250b3-2f1d-4295-a340-b8cc3a791279`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:44:41,024::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:41,024::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d1250b3-2f1d-4295-a340-b8cc3a791279`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108920832000', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:41,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d1250b3-2f1d-4295-a340-b8cc3a791279`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:44:41,024::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:44:41,025::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:44:41,025::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:44:41,025::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:44:41,025::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:44:41,025::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:44:41,025::task::993::Storage.TaskManager.Task::(_decref) Task=`8d1250b3-2f1d-4295-a340-b8cc3a791279`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:41,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:41,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116917 s, 87.6 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:44:43,177::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:43,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`598678cf-a6ba-4dfb-a308-46898b43809b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:43,177::logUtils::44::dispatcher::(wrapper) Run and protect: getFileStats(sdUUID='7c4cb940-6043-48fa-88f8-ab4762019c07', pattern='*.iso', caseSensitive=False, options=None) >Thread-46::DEBUG::2014-08-22 09:44:43,177::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.7c4cb940-6043-48fa-88f8-ab4762019c07`ReqID=`e15abf7f-ad6b-410a-9579-c5c39542f0d7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2295' at 'getFileStats' >Thread-46::DEBUG::2014-08-22 09:44:43,177::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:43,178::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:43,178::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.7c4cb940-6043-48fa-88f8-ab4762019c07`ReqID=`e15abf7f-ad6b-410a-9579-c5c39542f0d7`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:43,178::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`598678cf-a6ba-4dfb-a308-46898b43809b`::_resourcesAcquired: Storage.7c4cb940-6043-48fa-88f8-ab4762019c07 (shared) >Thread-46::DEBUG::2014-08-22 09:44:43,178::task::993::Storage.TaskManager.Task::(_decref) Task=`598678cf-a6ba-4dfb-a308-46898b43809b`::ref 1 aborting False >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) Message size is 205 >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) (1271) Finding callback 'listdir'... >Thread-17::DEBUG::2014-08-22 09:44:43,179::__init__::225::IOProcess::(_processLogs) (1271) Got request for method 'listdir' >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) (1271) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Sending response sized 210 >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Message size is 237 >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,180::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,181::__init__::225::IOProcess::(_processLogs) (1272) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,181::__init__::225::IOProcess::(_processLogs) (1272) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,181::__init__::225::IOProcess::(_processLogs) (1272) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,182::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,182::__init__::225::IOProcess::(_processLogs) Sending response sized 332 >Thread-17::DEBUG::2014-08-22 09:44:43,182::__init__::225::IOProcess::(_processLogs) Message size is 224 >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) (1273) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) (1273) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,183::__init__::225::IOProcess::(_processLogs) (1273) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Message size is 215 >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,184::__init__::225::IOProcess::(_processLogs) (1274) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,185::__init__::225::IOProcess::(_processLogs) (1274) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,185::__init__::225::IOProcess::(_processLogs) (1274) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,185::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,185::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) Message size is 240 >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) (1275) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) (1275) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) (1275) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) (1275) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,186::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,187::__init__::225::IOProcess::(_processLogs) Message size is 237 >Thread-17::DEBUG::2014-08-22 09:44:43,187::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,188::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,188::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,188::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,188::__init__::225::IOProcess::(_processLogs) (1276) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,188::__init__::225::IOProcess::(_processLogs) (1276) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,188::__init__::225::IOProcess::(_processLogs) (1276) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,189::__init__::225::IOProcess::(_processLogs) (1276) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,189::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,189::__init__::225::IOProcess::(_processLogs) Sending response sized 332 >Thread-17::DEBUG::2014-08-22 09:44:43,189::__init__::225::IOProcess::(_processLogs) Message size is 237 >Thread-17::DEBUG::2014-08-22 09:44:43,189::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) (1277) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) (1277) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) (1277) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) (1277) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,190::__init__::225::IOProcess::(_processLogs) Sending response sized 332 >Thread-17::DEBUG::2014-08-22 09:44:43,191::__init__::225::IOProcess::(_processLogs) Message size is 224 >Thread-17::DEBUG::2014-08-22 09:44:43,191::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,191::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,191::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,191::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) (1278) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) (1278) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) (1278) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) (1278) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) Message size is 224 >Thread-17::DEBUG::2014-08-22 09:44:43,192::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) (1279) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) (1279) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) (1279) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,193::__init__::225::IOProcess::(_processLogs) (1279) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,194::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,194::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,194::__init__::225::IOProcess::(_processLogs) Message size is 215 >Thread-17::DEBUG::2014-08-22 09:44:43,194::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,194::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) (1280) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) (1280) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) (1280) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) (1280) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,195::__init__::225::IOProcess::(_processLogs) Message size is 215 >Thread-17::DEBUG::2014-08-22 09:44:43,196::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,196::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,196::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) (1281) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) (1281) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) (1281) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) (1281) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) Message size is 240 >Thread-17::DEBUG::2014-08-22 09:44:43,197::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) (1282) Finding callback 'stat'... >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) (1282) Got request for method 'stat' >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) (1282) Building response >Thread-17::DEBUG::2014-08-22 09:44:43,198::__init__::225::IOProcess::(_processLogs) (1282) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:43,199::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:43,199::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-17::DEBUG::2014-08-22 09:44:43,199::__init__::225::IOProcess::(_processLogs) Message size is 240 >Thread-17::DEBUG::2014-08-22 09:44:43,199::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:43,199::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-46::INFO::2014-08-22 09:44:43,199::logUtils::47::dispatcher::(wrapper) Run and protect: getFileStats, Return response: {'fileStats': {u'FreeBSD-9.2-RELEASE-i386-disc1.iso': {'status': 0, 'ctime': '1407850000.0', 'size': '561233920'}, u'debian_7.iso': {'status': 0, 'ctime': '1407850000.0', 'size': '290455552'}, u'test_.__.___._.__.iso': {'status': 0, 'ctime': '1407860000.0', 'size': '290455552'}, u'FreeBSD-9.2-RELEASE-i386-bootonly.iso': {'status': 0, 'ctime': '1407850000.0', 'size': '158980096'}}} >Thread-17::DEBUG::2014-08-22 09:44:43,200::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-46::DEBUG::2014-08-22 09:44:43,200::task::1191::Storage.TaskManager.Task::(prepare) Task=`598678cf-a6ba-4dfb-a308-46898b43809b`::finished: {'fileStats': {u'FreeBSD-9.2-RELEASE-i386-disc1.iso': {'status': 0, 'ctime': '1407850000.0', 'size': '561233920'}, u'debian_7.iso': {'status': 0, 'ctime': '1407850000.0', 'size': '290455552'}, u'test_.__.___._.__.iso': {'status': 0, 'ctime': '1407860000.0', 'size': '290455552'}, u'FreeBSD-9.2-RELEASE-i386-bootonly.iso': {'status': 0, 'ctime': '1407850000.0', 'size': '158980096'}}} >Thread-46::DEBUG::2014-08-22 09:44:43,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`598678cf-a6ba-4dfb-a308-46898b43809b`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-08-22 09:44:43,201::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-46::DEBUG::2014-08-22 09:44:43,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07': < ResourceRef 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07', isValid: 'True' obj: 'None'>} >Thread-17::DEBUG::2014-08-22 09:44:43,201::__init__::225::IOProcess::(_processLogs) (1283) Finding callback 'stat'... >Thread-46::DEBUG::2014-08-22 09:44:43,201::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-08-22 09:44:43,202::__init__::225::IOProcess::(_processLogs) (1283) Got request for method 'stat' >Thread-46::DEBUG::2014-08-22 09:44:43,202::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' >Thread-17::DEBUG::2014-08-22 09:44:43,202::__init__::225::IOProcess::(_processLogs) (1283) Building response >Thread-46::DEBUG::2014-08-22 09:44:43,203::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' (0 active users) >Thread-17::DEBUG::2014-08-22 09:44:43,203::__init__::225::IOProcess::(_processLogs) (1283) Queuing response >Thread-46::DEBUG::2014-08-22 09:44:43,203::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-08-22 09:44:43,203::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-46::DEBUG::2014-08-22 09:44:43,204::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.7c4cb940-6043-48fa-88f8-ab4762019c07', Clearing records. >Thread-17::DEBUG::2014-08-22 09:44:43,204::__init__::225::IOProcess::(_processLogs) Sending response sized 331 >Thread-46::DEBUG::2014-08-22 09:44:43,204::task::993::Storage.TaskManager.Task::(_decref) Task=`598678cf-a6ba-4dfb-a308-46898b43809b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:43,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:43,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141618 s, 72.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:45,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:45,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118543 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:47,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:47,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118271 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:48,388::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:48,388::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:48,388::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:48,389::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:48,389::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:48,389::__init__::225::IOProcess::(_processLogs) (1284) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:48,389::__init__::225::IOProcess::(_processLogs) (1284) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:48,389::__init__::225::IOProcess::(_processLogs) (1284) Queuing response >Thread-21::DEBUG::2014-08-22 09:44:48,390::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:48,390::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:48,396::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:44:48,397::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000203314 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) (1285) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:48,398::__init__::225::IOProcess::(_processLogs) (1285) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:48,399::__init__::225::IOProcess::(_processLogs) (1285) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:48,400::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:48,400::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:44:48,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed44dae8-cde0-4c0c-8cbe-6cd3102b6dd4`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:48,761::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:44:48,761::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000203314', 'lastCheck': '0.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00139624', 'lastCheck': '9.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:48,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed44dae8-cde0-4c0c-8cbe-6cd3102b6dd4`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000203314', 'lastCheck': '0.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00139624', 'lastCheck': '9.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:44:48,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed44dae8-cde0-4c0c-8cbe-6cd3102b6dd4`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:48,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:48,761::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:48,762::task::993::Storage.TaskManager.Task::(_decref) Task=`ed44dae8-cde0-4c0c-8cbe-6cd3102b6dd4`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) (1286) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:49,200::__init__::225::IOProcess::(_processLogs) (1286) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:49,201::__init__::225::IOProcess::(_processLogs) (1286) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:49,201::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:49,201::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:44:49,201::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:44:49,207::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234752 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:49,208::__init__::225::IOProcess::(_processLogs) (1287) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:49,209::__init__::225::IOProcess::(_processLogs) (1287) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:49,209::__init__::225::IOProcess::(_processLogs) DEBUG|(1287) Building response >Thread-17::DEBUG::2014-08-22 09:44:49,210::__init__::225::IOProcess::(_processLogs) (1287) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:49,210::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:49,210::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:49,210::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:49,210::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:44:49,210::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) (1288) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) (1288) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:49,211::__init__::225::IOProcess::(_processLogs) (1288) Building response >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) (1288) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:49,212::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) (1289) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) (1289) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) (1289) Building response >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) (1289) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:49,213::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:49,214::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:49,214::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:44:49,214::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:49,214::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) (1290) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) (1290) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) (1290) Building response >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) (1290) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:49,215::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:44:49,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:49,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118212 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:44:51,079::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:44:51,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ef7182b-c5de-4f30-a2ee-abe0afbba2d6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:44:51,079::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:44:51,079::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:51,079::task::1191::Storage.TaskManager.Task::(prepare) Task=`8ef7182b-c5de-4f30-a2ee-abe0afbba2d6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:44:51,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ef7182b-c5de-4f30-a2ee-abe0afbba2d6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:44:51,080::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:44:51,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:44:51,080::task::993::Storage.TaskManager.Task::(_decref) Task=`8ef7182b-c5de-4f30-a2ee-abe0afbba2d6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:44:51,142::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:44:51,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`f32f2bda-063b-46f5-89d8-f0219f954cde`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:44:51,143::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:44:51,143::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ff0cd246-aaab-4794-bde3-ccfdf864e0e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:44:51,143::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:44:51,143::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:44:51,143::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ff0cd246-aaab-4794-bde3-ccfdf864e0e2`::Granted request >Thread-46::DEBUG::2014-08-22 09:44:51,144::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f32f2bda-063b-46f5-89d8-f0219f954cde`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:44:51,144::task::993::Storage.TaskManager.Task::(_decref) Task=`f32f2bda-063b-46f5-89d8-f0219f954cde`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:44:51,146::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:51,146::task::1191::Storage.TaskManager.Task::(prepare) Task=`f32f2bda-063b-46f5-89d8-f0219f954cde`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:44:51,146::task::595::Storage.TaskManager.Task::(_updateState) Task=`f32f2bda-063b-46f5-89d8-f0219f954cde`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:44:51,146::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:44:51,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:44:51,146::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:44:51,146::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:44:51,146::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:44:51,146::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:44:51,146::task::993::Storage.TaskManager.Task::(_decref) Task=`f32f2bda-063b-46f5-89d8-f0219f954cde`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:44:51,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:51,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116653 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:53,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:53,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116522 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:55,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:55,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116047 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:44:57,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:57,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011402 s, 89.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:58,400::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:58,400::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:58,400::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:58,401::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:58,401::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:58,401::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:58,401::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:58,401::__init__::225::IOProcess::(_processLogs) (1291) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:58,401::__init__::225::IOProcess::(_processLogs) (1291) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:58,402::__init__::225::IOProcess::(_processLogs) DEBUG|(1291) Building response >Thread-21::DEBUG::2014-08-22 09:44:58,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:58,402::__init__::225::IOProcess::(_processLogs) (1291) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:58,408::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:58,409::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:44:58,410::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178595 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:58,410::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) (1292) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) (1292) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) (1292) Building response >Thread-17::DEBUG::2014-08-22 09:44:58,411::__init__::225::IOProcess::(_processLogs) (1292) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:58,412::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:59,215::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:59,215::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:59,215::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:59,215::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:59,216::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:59,216::__init__::225::IOProcess::(_processLogs) (1293) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:59,216::__init__::225::IOProcess::(_processLogs) (1293) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:59,216::__init__::225::IOProcess::(_processLogs) (1293) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:59,217::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:44:59,217::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:44:59,217::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:44:59,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216263 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) (1294) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:44:59,225::__init__::225::IOProcess::(_processLogs) (1294) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:44:59,226::__init__::225::IOProcess::(_processLogs) DEBUG|(1294) Building response >Thread-17::DEBUG::2014-08-22 09:44:59,227::__init__::225::IOProcess::(_processLogs) (1294) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:59,227::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:59,227::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:44:59,227::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:59,227::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:44:59,227::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) (1295) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) (1295) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) (1295) Building response >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) (1295) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:59,228::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:59,229::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:59,229::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:59,229::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:44:59,229::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:59,229::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) (1296) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) (1296) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) (1296) Building response >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) (1296) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:44:59,230::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:44:59,231::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:44:59,231::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:44:59,231::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:44:59,231::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) (1297) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) (1297) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) (1297) Building response >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) (1297) Queuing response >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:44:59,232::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:44:59,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:44:59,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120695 s, 84.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:01,198::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:45:01,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`bee578cb-5e7c-478a-94b1-0e740c41d919`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:01,199::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:45:01,199::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:01,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`bee578cb-5e7c-478a-94b1-0e740c41d919`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:01,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`bee578cb-5e7c-478a-94b1-0e740c41d919`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:01,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:01,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:01,199::task::993::Storage.TaskManager.Task::(_decref) Task=`bee578cb-5e7c-478a-94b1-0e740c41d919`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:45:01,293::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:45:01,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec3c666f-79ae-46c5-ac54-a48262809c1b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:45:01,293::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:45:01,294::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`516d311b-36c5-4c99-97ea-34e389b23bbb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:45:01,294::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:45:01,294::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:45:01,294::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`516d311b-36c5-4c99-97ea-34e389b23bbb`::Granted request >Thread-46::DEBUG::2014-08-22 09:45:01,294::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ec3c666f-79ae-46c5-ac54-a48262809c1b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:45:01,294::task::993::Storage.TaskManager.Task::(_decref) Task=`ec3c666f-79ae-46c5-ac54-a48262809c1b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:45:01,296::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:01,296::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec3c666f-79ae-46c5-ac54-a48262809c1b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:01,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec3c666f-79ae-46c5-ac54-a48262809c1b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:45:01,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:45:01,297::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:45:01,297::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:45:01,297::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:45:01,297::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:45:01,297::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:45:01,297::task::993::Storage.TaskManager.Task::(_decref) Task=`ec3c666f-79ae-46c5-ac54-a48262809c1b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:01,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:01,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115674 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:03,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:04,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113887 s, 89.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:04,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5ae3796-ee31-4b68-bf08-94ac042e1d3c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:04,006::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:45:04,007::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178595', 'lastCheck': '5.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216263', 'lastCheck': '4.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:04,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5ae3796-ee31-4b68-bf08-94ac042e1d3c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178595', 'lastCheck': '5.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000216263', 'lastCheck': '4.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:04,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5ae3796-ee31-4b68-bf08-94ac042e1d3c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:04,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:04,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:04,009::task::993::Storage.TaskManager.Task::(_decref) Task=`b5ae3796-ee31-4b68-bf08-94ac042e1d3c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:06,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:06,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113813 s, 90.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:08,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:08,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117357 s, 87.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:08,413::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) (1298) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) (1298) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) (1298) Building response >Thread-17::DEBUG::2014-08-22 09:45:08,414::__init__::225::IOProcess::(_processLogs) (1298) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:08,415::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:45:08,416::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:45:08,416::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:45:08,422::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000260245 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:08,423::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:08,424::__init__::225::IOProcess::(_processLogs) (1299) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:08,424::__init__::225::IOProcess::(_processLogs) (1299) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:08,424::__init__::225::IOProcess::(_processLogs) (1299) Building response >Thread-17::DEBUG::2014-08-22 09:45:08,424::__init__::225::IOProcess::(_processLogs) (1299) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:08,424::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:45:08,425::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:09,232::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:09,232::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:09,232::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:09,232::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:09,232::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:09,232::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:09,233::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:09,233::__init__::225::IOProcess::(_processLogs) (1300) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:09,233::__init__::225::IOProcess::(_processLogs) (1300) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:09,233::__init__::225::IOProcess::(_processLogs) DEBUG|(1300) Building response >Thread-17::DEBUG::2014-08-22 09:45:09,234::__init__::225::IOProcess::(_processLogs) (1300) Queuing response >Thread-22::DEBUG::2014-08-22 09:45:09,234::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:09,234::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:09,240::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:45:09,240::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214585 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) (1301) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:09,241::__init__::225::IOProcess::(_processLogs) (1301) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:09,242::__init__::225::IOProcess::(_processLogs) (1301) Building response >Thread-17::DEBUG::2014-08-22 09:45:09,242::__init__::225::IOProcess::(_processLogs) (1301) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:09,242::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:45:09,243::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:09,243::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:09,243::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:45:09,243::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) (1302) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) (1302) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) (1302) Building response >Thread-17::DEBUG::2014-08-22 09:45:09,244::__init__::225::IOProcess::(_processLogs) (1302) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:09,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:09,245::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:09,245::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:09,245::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:45:09,245::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:09,245::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) (1303) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) (1303) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) (1303) Building response >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) (1303) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:09,246::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:09,247::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:09,247::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:45:09,247::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:09,247::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:09,247::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) (1304) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) (1304) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) (1304) Building response >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) (1304) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:09,248::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:45:10,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:10,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116241 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:11,427::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:45:11,427::task::595::Storage.TaskManager.Task::(_updateState) Task=`533dbe94-1a70-4e83-900a-c98ec71567d6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:11,427::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:45:11,428::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:11,428::task::1191::Storage.TaskManager.Task::(prepare) Task=`533dbe94-1a70-4e83-900a-c98ec71567d6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:11,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`533dbe94-1a70-4e83-900a-c98ec71567d6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:11,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:11,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:11,428::task::993::Storage.TaskManager.Task::(_decref) Task=`533dbe94-1a70-4e83-900a-c98ec71567d6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:45:11,453::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:45:11,453::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab228fee-ef47-46eb-b0e7-9fa3f496b4d7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:45:11,453::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:45:11,453::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`34b95904-59b7-41ba-8160-1a175b7cc692`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:45:11,453::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:45:11,453::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:45:11,454::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`34b95904-59b7-41ba-8160-1a175b7cc692`::Granted request >Thread-46::DEBUG::2014-08-22 09:45:11,454::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ab228fee-ef47-46eb-b0e7-9fa3f496b4d7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:45:11,454::task::993::Storage.TaskManager.Task::(_decref) Task=`ab228fee-ef47-46eb-b0e7-9fa3f496b4d7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:45:11,456::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:11,456::task::1191::Storage.TaskManager.Task::(prepare) Task=`ab228fee-ef47-46eb-b0e7-9fa3f496b4d7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:11,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab228fee-ef47-46eb-b0e7-9fa3f496b4d7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:45:11,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:45:11,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:45:11,456::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:45:11,456::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:45:11,456::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:45:11,457::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:45:11,457::task::993::Storage.TaskManager.Task::(_decref) Task=`ab228fee-ef47-46eb-b0e7-9fa3f496b4d7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:12,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:12,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200489 s, 51.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:14,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:14,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117343 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:16,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:16,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011915 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:18,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:18,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117014 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:18,426::__init__::225::IOProcess::(_processLogs) (1305) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:18,427::__init__::225::IOProcess::(_processLogs) (1305) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:18,427::__init__::225::IOProcess::(_processLogs) DEBUG|(1305) Building response >Thread-21::DEBUG::2014-08-22 09:45:18,428::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:18,428::__init__::225::IOProcess::(_processLogs) (1305) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:18,434::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:18,434::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:45:18,434::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000161174 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:18,435::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:18,435::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:18,436::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:18,436::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:18,436::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) (1306) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) (1306) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) (1306) Building response >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) (1306) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:18,437::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:19,248::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:19,248::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:19,248::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:19,248::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:19,249::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:19,249::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:19,249::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:19,249::__init__::225::IOProcess::(_processLogs) (1307) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:19,249::__init__::225::IOProcess::(_processLogs) (1307) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:19,249::__init__::225::IOProcess::(_processLogs) DEBUG|(1307) Building response >Thread-17::DEBUG::2014-08-22 09:45:19,250::__init__::225::IOProcess::(_processLogs) (1307) Queuing response >Thread-22::DEBUG::2014-08-22 09:45:19,250::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:19,250::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:19,256::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:45:19,257::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000203598 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:19,257::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:19,257::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) (1308) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:19,258::__init__::225::IOProcess::(_processLogs) (1308) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:19,259::__init__::225::IOProcess::(_processLogs) DEBUG|(1308) Building response >Thread-17::DEBUG::2014-08-22 09:45:19,259::__init__::225::IOProcess::(_processLogs) (1308) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:19,259::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:19,259::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:19,260::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:19,260::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:45:19,260::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:19,260::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:19,260::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:19,260::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) (1309) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) (1309) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) (1309) Building response >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) (1309) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:45:19,261::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) (1310) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) (1310) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) (1310) Building response >Thread-17::DEBUG::2014-08-22 09:45:19,262::__init__::225::IOProcess::(_processLogs) (1310) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:19,263::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) (1311) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) (1311) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) (1311) Building response >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) (1311) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:19,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:19,265::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:45:19,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8c1e557-3ad8-4061-8d14-1244fce9404f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:19,294::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:45:19,294::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000161174', 'lastCheck': '0.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000203598', 'lastCheck': '0.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:19,294::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8c1e557-3ad8-4061-8d14-1244fce9404f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000161174', 'lastCheck': '0.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000203598', 'lastCheck': '0.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:19,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8c1e557-3ad8-4061-8d14-1244fce9404f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:19,294::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:19,294::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:19,295::task::993::Storage.TaskManager.Task::(_decref) Task=`a8c1e557-3ad8-4061-8d14-1244fce9404f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:20,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:20,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116181 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:21,532::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:45:21,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fd853e7-13a2-46a5-95f5-2c05026ff7f2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:21,532::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:45:21,532::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:21,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`6fd853e7-13a2-46a5-95f5-2c05026ff7f2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:21,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fd853e7-13a2-46a5-95f5-2c05026ff7f2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:21,533::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:21,533::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:21,533::task::993::Storage.TaskManager.Task::(_decref) Task=`6fd853e7-13a2-46a5-95f5-2c05026ff7f2`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:45:21,607::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:45:21,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4e023ce-9708-4487-81f3-cf7fbc406b6d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:45:21,608::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:45:21,608::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6cf0838e-9757-4f8d-9393-3b0b2c08a9b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:45:21,608::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:45:21,608::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:45:21,608::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6cf0838e-9757-4f8d-9393-3b0b2c08a9b0`::Granted request >Thread-46::DEBUG::2014-08-22 09:45:21,608::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a4e023ce-9708-4487-81f3-cf7fbc406b6d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:45:21,609::task::993::Storage.TaskManager.Task::(_decref) Task=`a4e023ce-9708-4487-81f3-cf7fbc406b6d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:45:21,610::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:21,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4e023ce-9708-4487-81f3-cf7fbc406b6d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:21,611::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4e023ce-9708-4487-81f3-cf7fbc406b6d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:45:21,611::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:45:21,611::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:45:21,611::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:45:21,611::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:45:21,611::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:45:21,611::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:45:21,611::task::993::Storage.TaskManager.Task::(_decref) Task=`a4e023ce-9708-4487-81f3-cf7fbc406b6d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:22,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:22,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116515 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:24,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:24,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011394 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:26,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:26,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193729 s, 52.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:28,437::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:28,437::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:28,437::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:28,438::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:45:28,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:28,439::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:28,444::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:28,445::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:28,446::__init__::225::IOProcess::(_processLogs) (1312) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:45:28,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196876 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:28,447::__init__::225::IOProcess::(_processLogs) (1312) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:28,447::__init__::225::IOProcess::(_processLogs) (1312) Building response >Thread-17::DEBUG::2014-08-22 09:45:28,447::__init__::225::IOProcess::(_processLogs) (1312) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:28,447::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:28,447::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) (1313) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:28,448::__init__::225::IOProcess::(_processLogs) (1313) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:28,449::__init__::225::IOProcess::(_processLogs) DEBUG|(1313) Building response >Thread-17::DEBUG::2014-08-22 09:45:28,450::__init__::225::IOProcess::(_processLogs) (1313) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:28,450::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:28,450::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:45:28,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:28,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011434 s, 89.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:29,264::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) (1314) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) (1314) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) (1314) Building response >Thread-17::DEBUG::2014-08-22 09:45:29,265::__init__::225::IOProcess::(_processLogs) (1314) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:29,266::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:29,266::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:45:29,267::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:45:29,273::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000248252 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:29,273::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) (1315) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:29,274::__init__::225::IOProcess::(_processLogs) (1315) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:29,275::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1315) Building response >Thread-17::DEBUG::2014-08-22 09:45:29,275::__init__::225::IOProcess::(_processLogs) (1315) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:29,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:29,275::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:29,276::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:29,276::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:45:29,276::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:29,276::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) (1316) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) (1316) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) (1316) Building response >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) (1316) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:29,277::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:29,278::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:29,278::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:45:29,278::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:29,279::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:29,279::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:29,279::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:29,279::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:29,279::__init__::225::IOProcess::(_processLogs) (1317) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) (1317) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) (1317) Building response >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) (1317) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:45:29,280::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:29,281::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:29,281::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:29,281::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:29,281::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:29,281::__init__::225::IOProcess::(_processLogs) (1318) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:29,282::__init__::225::IOProcess::(_processLogs) (1318) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:29,282::__init__::225::IOProcess::(_processLogs) (1318) Building response >Thread-17::DEBUG::2014-08-22 09:45:29,282::__init__::225::IOProcess::(_processLogs) (1318) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:29,282::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:29,282::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:45:30,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:30,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117179 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:31,660::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:45:31,660::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecb3e0e8-4938-4b43-a7b1-25333cd25675`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:31,660::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:45:31,660::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:31,661::task::1191::Storage.TaskManager.Task::(prepare) Task=`ecb3e0e8-4938-4b43-a7b1-25333cd25675`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:31,661::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecb3e0e8-4938-4b43-a7b1-25333cd25675`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:31,661::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:31,661::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:31,661::task::993::Storage.TaskManager.Task::(_decref) Task=`ecb3e0e8-4938-4b43-a7b1-25333cd25675`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:45:31,702::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:45:31,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`77b7d52e-d941-4fdf-9501-5e9c2fce9ef9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:45:31,702::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:45:31,703::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ddfd8792-d1af-46df-ab4b-842ae95ce328`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:45:31,703::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:45:31,703::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:45:31,703::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ddfd8792-d1af-46df-ab4b-842ae95ce328`::Granted request >Thread-46::DEBUG::2014-08-22 09:45:31,703::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`77b7d52e-d941-4fdf-9501-5e9c2fce9ef9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:45:31,703::task::993::Storage.TaskManager.Task::(_decref) Task=`77b7d52e-d941-4fdf-9501-5e9c2fce9ef9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:45:31,705::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:31,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`77b7d52e-d941-4fdf-9501-5e9c2fce9ef9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:31,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`77b7d52e-d941-4fdf-9501-5e9c2fce9ef9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:45:31,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:45:31,706::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:45:31,706::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:45:31,706::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:45:31,706::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:45:31,706::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:45:31,706::task::993::Storage.TaskManager.Task::(_decref) Task=`77b7d52e-d941-4fdf-9501-5e9c2fce9ef9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:32,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:32,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217709 s, 47.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:34,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`da308341-e223-420e-8c93-845e89f3bab5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:34,532::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:45:34,533::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196876', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000248252', 'lastCheck': '5.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:34,533::task::1191::Storage.TaskManager.Task::(prepare) Task=`da308341-e223-420e-8c93-845e89f3bab5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000196876', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000248252', 'lastCheck': '5.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:34,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`da308341-e223-420e-8c93-845e89f3bab5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:34,533::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:34,533::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:34,533::task::993::Storage.TaskManager.Task::(_decref) Task=`da308341-e223-420e-8c93-845e89f3bab5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:34,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:34,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117278 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:36,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:36,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118581 s, 86.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:38,450::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:38,451::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:38,451::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:38,451::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:45:38,452::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:38,452::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:38,458::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:45:38,459::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000139547 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:38,459::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) (1319) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) (1319) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) (1319) Building response >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) (1319) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:38,461::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:38,462::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:38,462::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:38,462::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:38,462::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:38,462::__init__::225::IOProcess::(_processLogs) (1320) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:38,462::__init__::225::IOProcess::(_processLogs) (1320) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:38,463::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1320) Building response >Thread-17::DEBUG::2014-08-22 09:45:38,464::__init__::225::IOProcess::(_processLogs) (1320) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:38,464::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:38,464::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:45:38,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:38,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117918 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) (1321) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:39,282::__init__::225::IOProcess::(_processLogs) (1321) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:39,283::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1321) Building response >Thread-17::DEBUG::2014-08-22 09:45:39,283::__init__::225::IOProcess::(_processLogs) (1321) Queuing response >Thread-22::DEBUG::2014-08-22 09:45:39,284::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:39,284::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:39,290::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:45:39,291::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000217485 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:39,291::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) (1322) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) (1322) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) (1322) Building response >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) (1322) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:39,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:39,293::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:39,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:39,294::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:45:39,294::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:39,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) (1323) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) (1323) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) (1323) Building response >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) (1323) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:39,295::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:39,296::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:39,296::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:45:39,296::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:39,296::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:39,296::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) (1324) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) (1324) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) (1324) Building response >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) (1324) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:45:39,297::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:39,298::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:39,298::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:39,298::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) (1325) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) (1325) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) (1325) Building response >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) (1325) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:39,299::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:45:40,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:41,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118165 s, 86.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:41,754::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:45:41,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe9feaf2-f385-447b-b136-85eaef65f606`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:41,755::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:45:41,755::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:41,755::task::1191::Storage.TaskManager.Task::(prepare) Task=`fe9feaf2-f385-447b-b136-85eaef65f606`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:41,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe9feaf2-f385-447b-b136-85eaef65f606`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:41,755::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:41,755::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:41,755::task::993::Storage.TaskManager.Task::(_decref) Task=`fe9feaf2-f385-447b-b136-85eaef65f606`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:45:41,780::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:45:41,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`69fdf733-fbb0-44bc-b7dd-5cc00a4cc9a1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:45:41,780::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:45:41,781::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8705538f-a078-459a-a66d-f05f6395adaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:45:41,781::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:45:41,781::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:45:41,781::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8705538f-a078-459a-a66d-f05f6395adaf`::Granted request >Thread-46::DEBUG::2014-08-22 09:45:41,781::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`69fdf733-fbb0-44bc-b7dd-5cc00a4cc9a1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:45:41,781::task::993::Storage.TaskManager.Task::(_decref) Task=`69fdf733-fbb0-44bc-b7dd-5cc00a4cc9a1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:45:41,783::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:41,783::task::1191::Storage.TaskManager.Task::(prepare) Task=`69fdf733-fbb0-44bc-b7dd-5cc00a4cc9a1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108918734848', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:41,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`69fdf733-fbb0-44bc-b7dd-5cc00a4cc9a1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:45:41,784::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:45:41,784::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:45:41,784::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:45:41,784::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:45:41,784::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:45:41,784::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:45:41,784::task::993::Storage.TaskManager.Task::(_decref) Task=`69fdf733-fbb0-44bc-b7dd-5cc00a4cc9a1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:43,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:43,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119083 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:45,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:45,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120773 s, 84.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:47,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:47,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133237 s, 76.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:48,464::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:48,464::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:48,464::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:48,464::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:48,465::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:48,465::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:48,465::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:48,465::__init__::225::IOProcess::(_processLogs) (1326) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:48,465::__init__::225::IOProcess::(_processLogs) (1326) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:48,465::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1326) Building response >Thread-21::DEBUG::2014-08-22 09:45:48,466::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:48,466::__init__::225::IOProcess::(_processLogs) (1326) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:48,472::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:45:48,473::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000259362 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:48,473::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) (1327) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:48,475::__init__::225::IOProcess::(_processLogs) (1327) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:48,476::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1327) Building response >Thread-17::DEBUG::2014-08-22 09:45:48,477::__init__::225::IOProcess::(_processLogs) (1327) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:48,477::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:48,477::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:45:49,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:49,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118404 s, 86.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:49,299::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:49,299::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:49,299::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:49,299::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:49,299::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) (1328) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) (1328) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) (1328) Building response >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) (1328) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:49,300::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:45:49,301::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:45:49,307::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223864 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:49,307::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:49,307::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:49,308::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) (1329) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) (1329) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) (1329) Building response >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) (1329) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:49,309::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:49,310::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:49,310::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:45:49,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:49,310::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:49,310::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) (1330) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) (1330) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) (1330) Building response >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) (1330) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:49,311::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:45:49,312::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:49,312::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:49,312::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:49,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:49,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:49,312::__init__::225::IOProcess::(_processLogs) (1331) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) (1331) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) (1331) Building response >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) (1331) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:45:49,313::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:49,314::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:49,314::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:49,314::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:49,314::__init__::225::IOProcess::(_processLogs) (1332) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:49,315::__init__::225::IOProcess::(_processLogs) (1332) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:49,315::__init__::225::IOProcess::(_processLogs) (1332) Building response >Thread-17::DEBUG::2014-08-22 09:45:49,315::__init__::225::IOProcess::(_processLogs) (1332) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:49,315::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:49,315::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:45:49,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`f84985e1-8c4e-4330-b227-55e472f6d309`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:49,704::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:45:49,705::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000259362', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223864', 'lastCheck': '0.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:49,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`f84985e1-8c4e-4330-b227-55e472f6d309`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000259362', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223864', 'lastCheck': '0.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:45:49,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`f84985e1-8c4e-4330-b227-55e472f6d309`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:49,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:49,705::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:49,705::task::993::Storage.TaskManager.Task::(_decref) Task=`f84985e1-8c4e-4330-b227-55e472f6d309`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:51,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:51,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122067 s, 83.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:45:51,824::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:45:51,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`83b9b87a-4d6d-4493-bc3d-6bad837985f5`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:45:51,825::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:45:51,825::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:51,825::task::1191::Storage.TaskManager.Task::(prepare) Task=`83b9b87a-4d6d-4493-bc3d-6bad837985f5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:45:51,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`83b9b87a-4d6d-4493-bc3d-6bad837985f5`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:45:51,825::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:45:51,825::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:45:51,825::task::993::Storage.TaskManager.Task::(_decref) Task=`83b9b87a-4d6d-4493-bc3d-6bad837985f5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:45:51,860::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:45:51,860::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9517db4-6fe2-4446-bce8-c96d92868f77`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:45:51,860::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:45:51,860::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c21c0cdf-2177-41cd-84b1-8c0ece3f0f95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:45:51,861::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:45:51,861::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:45:51,861::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c21c0cdf-2177-41cd-84b1-8c0ece3f0f95`::Granted request >Thread-46::DEBUG::2014-08-22 09:45:51,861::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b9517db4-6fe2-4446-bce8-c96d92868f77`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:45:51,861::task::993::Storage.TaskManager.Task::(_decref) Task=`b9517db4-6fe2-4446-bce8-c96d92868f77`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:45:51,863::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:51,863::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9517db4-6fe2-4446-bce8-c96d92868f77`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:45:51,863::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9517db4-6fe2-4446-bce8-c96d92868f77`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:45:51,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:45:51,863::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:45:51,864::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:45:51,864::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:45:51,864::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:45:51,864::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:45:51,864::task::993::Storage.TaskManager.Task::(_decref) Task=`b9517db4-6fe2-4446-bce8-c96d92868f77`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:45:53,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:53,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116475 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:55,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:55,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115325 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:45:57,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:57,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123815 s, 82.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:58,477::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:58,477::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) (1333) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:58,478::__init__::225::IOProcess::(_processLogs) (1333) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:58,479::__init__::225::IOProcess::(_processLogs) DEBUG|(1333) Building response >Thread-21::DEBUG::2014-08-22 09:45:58,479::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:45:58,479::__init__::225::IOProcess::(_processLogs) (1333) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:58,486::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:45:58,487::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189078 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:58,487::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:58,488::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:58,489::__init__::225::IOProcess::(_processLogs) (1334) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:58,489::__init__::225::IOProcess::(_processLogs) (1334) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:58,489::__init__::225::IOProcess::(_processLogs) DEBUG|(1334) Building response >Thread-17::DEBUG::2014-08-22 09:45:58,490::__init__::225::IOProcess::(_processLogs) (1334) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:58,490::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:58,490::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:59,315::__init__::225::IOProcess::(_processLogs) (1335) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:59,316::__init__::225::IOProcess::(_processLogs) (1335) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:59,316::__init__::225::IOProcess::(_processLogs) (1335) Building response >Thread-17::DEBUG::2014-08-22 09:45:59,316::__init__::225::IOProcess::(_processLogs) (1335) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:59,316::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:59,316::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:45:59,317::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:45:59,323::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000258494 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:45:59,323::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:59,323::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:45:59,324::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:59,324::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) (1336) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) (1336) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) (1336) Building response >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) (1336) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:59,325::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:59,326::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:45:59,326::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:59,326::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:45:59,326::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:59,326::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) (1337) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) (1337) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) (1337) Building response >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) (1337) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:59,327::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:59,328::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:45:59,328::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:59,328::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:59,328::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:59,328::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) (1338) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) (1338) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) (1338) Building response >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) (1338) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:45:59,329::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:45:59,330::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:45:59,330::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:45:59,330::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:45:59,330::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:45:59,331::__init__::225::IOProcess::(_processLogs) (1339) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:45:59,331::__init__::225::IOProcess::(_processLogs) (1339) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:45:59,331::__init__::225::IOProcess::(_processLogs) (1339) Building response >Thread-17::DEBUG::2014-08-22 09:45:59,331::__init__::225::IOProcess::(_processLogs) (1339) Queuing response >Thread-17::DEBUG::2014-08-22 09:45:59,331::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:45:59,331::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:45:59,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:45:59,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117719 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:01,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:01,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119238 s, 85.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:01,913::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:46:01,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`1390ae88-f5ce-4874-b6b8-665b7ad6a12c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:01,914::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:46:01,914::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:01,914::task::1191::Storage.TaskManager.Task::(prepare) Task=`1390ae88-f5ce-4874-b6b8-665b7ad6a12c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:01,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`1390ae88-f5ce-4874-b6b8-665b7ad6a12c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:01,914::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:01,914::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:01,915::task::993::Storage.TaskManager.Task::(_decref) Task=`1390ae88-f5ce-4874-b6b8-665b7ad6a12c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:46:01,947::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:46:01,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`baf74f49-7f1e-4203-b1a4-d7f50e589a2b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:46:01,947::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:46:01,948::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2fef465e-4a2b-4f7f-9619-c55068462377`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:46:01,948::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:46:01,948::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:46:01,948::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2fef465e-4a2b-4f7f-9619-c55068462377`::Granted request >Thread-46::DEBUG::2014-08-22 09:46:01,948::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`baf74f49-7f1e-4203-b1a4-d7f50e589a2b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:46:01,948::task::993::Storage.TaskManager.Task::(_decref) Task=`baf74f49-7f1e-4203-b1a4-d7f50e589a2b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:46:01,950::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:01,950::task::1191::Storage.TaskManager.Task::(prepare) Task=`baf74f49-7f1e-4203-b1a4-d7f50e589a2b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:01,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`baf74f49-7f1e-4203-b1a4-d7f50e589a2b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:46:01,951::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:46:01,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:46:01,951::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:46:01,951::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:46:01,951::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:46:01,951::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:46:01,951::task::993::Storage.TaskManager.Task::(_decref) Task=`baf74f49-7f1e-4203-b1a4-d7f50e589a2b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:03,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:03,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116214 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:04,994::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cd54c2d-b6b2-4c2d-bb45-5f3f5e696cfb`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:04,994::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:46:04,994::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189078', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000258494', 'lastCheck': '5.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:04,994::task::1191::Storage.TaskManager.Task::(prepare) Task=`5cd54c2d-b6b2-4c2d-bb45-5f3f5e696cfb`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189078', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000258494', 'lastCheck': '5.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:04,994::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cd54c2d-b6b2-4c2d-bb45-5f3f5e696cfb`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:04,994::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:04,995::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:04,995::task::993::Storage.TaskManager.Task::(_decref) Task=`5cd54c2d-b6b2-4c2d-bb45-5f3f5e696cfb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:05,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:05,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115094 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:07,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:07,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011616 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) (1340) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:08,491::__init__::225::IOProcess::(_processLogs) (1340) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:08,492::__init__::225::IOProcess::(_processLogs) DEBUG|(1340) Building response >Thread-21::DEBUG::2014-08-22 09:46:08,493::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:08,493::__init__::225::IOProcess::(_processLogs) (1340) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:08,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:46:08,500::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000155828 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:08,500::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:08,501::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:08,501::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:08,501::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:08,502::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:08,502::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:08,502::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:08,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:08,502::__init__::225::IOProcess::(_processLogs) (1341) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:08,502::__init__::225::IOProcess::(_processLogs) (1341) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:08,503::__init__::225::IOProcess::(_processLogs) DEBUG|(1341) Building response >Thread-17::DEBUG::2014-08-22 09:46:08,503::__init__::225::IOProcess::(_processLogs) (1341) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:08,504::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:08,504::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:09,331::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:09,331::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:09,331::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:09,331::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:46:09,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:09,332::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:09,338::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:46:09,339::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000179369 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:09,339::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:09,340::__init__::225::IOProcess::(_processLogs) (1342) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:09,340::__init__::225::IOProcess::(_processLogs) (1342) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:09,340::__init__::225::IOProcess::(_processLogs) (1342) Building response >Thread-17::DEBUG::2014-08-22 09:46:09,340::__init__::225::IOProcess::(_processLogs) (1342) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:09,340::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:09,340::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) (1343) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:09,341::__init__::225::IOProcess::(_processLogs) (1343) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:09,342::__init__::225::IOProcess::(_processLogs) DEBUG|(1343) Building response >Thread-17::DEBUG::2014-08-22 09:46:09,343::__init__::225::IOProcess::(_processLogs) (1343) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:09,343::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:09,343::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:09,343::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:09,343::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:46:09,343::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) (1344) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) (1344) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) (1344) Building response >Thread-17::DEBUG::2014-08-22 09:46:09,344::__init__::225::IOProcess::(_processLogs) (1344) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:09,345::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:09,345::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:09,345::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:09,345::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:46:09,345::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:09,345::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) (1345) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) (1345) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) (1345) Building response >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) (1345) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:09,346::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:09,347::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:46:09,347::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:09,347::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:09,347::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) (1346) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) (1346) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) (1346) Building response >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) (1346) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:09,348::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:46:09,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:09,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121646 s, 84.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:11,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:11,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114148 s, 89.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:12,025::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:46:12,025::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c6d0d56-8278-4041-bab3-73cb9bbd5bdc`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:12,025::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:46:12,025::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:12,025::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c6d0d56-8278-4041-bab3-73cb9bbd5bdc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:12,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c6d0d56-8278-4041-bab3-73cb9bbd5bdc`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:12,026::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:12,026::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:12,026::task::993::Storage.TaskManager.Task::(_decref) Task=`6c6d0d56-8278-4041-bab3-73cb9bbd5bdc`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:46:12,083::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:46:12,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0014421-aef7-4efa-8471-2ad0508a7a1b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:46:12,083::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:46:12,084::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`337c001a-21d1-4bc3-866e-b1b3d8dd58ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:46:12,084::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:46:12,084::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:46:12,084::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`337c001a-21d1-4bc3-866e-b1b3d8dd58ea`::Granted request >Thread-46::DEBUG::2014-08-22 09:46:12,084::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d0014421-aef7-4efa-8471-2ad0508a7a1b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:46:12,084::task::993::Storage.TaskManager.Task::(_decref) Task=`d0014421-aef7-4efa-8471-2ad0508a7a1b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:46:12,086::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:12,086::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0014421-aef7-4efa-8471-2ad0508a7a1b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:12,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0014421-aef7-4efa-8471-2ad0508a7a1b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:46:12,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:46:12,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:46:12,087::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:46:12,087::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:46:12,087::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:46:12,087::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:46:12,087::task::993::Storage.TaskManager.Task::(_decref) Task=`d0014421-aef7-4efa-8471-2ad0508a7a1b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:13,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:13,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116714 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:15,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:15,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118741 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:17,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:17,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011647 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:18,504::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:18,504::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:18,504::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:18,504::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:18,504::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:18,504::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:18,505::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:18,505::__init__::225::IOProcess::(_processLogs) (1347) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:18,505::__init__::225::IOProcess::(_processLogs) (1347) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:18,505::__init__::225::IOProcess::(_processLogs) DEBUG|(1347) Building response >Thread-21::DEBUG::2014-08-22 09:46:18,506::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:18,506::__init__::225::IOProcess::(_processLogs) (1347) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:18,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:46:18,513::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199868 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:18,513::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:18,514::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:18,514::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) (1348) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:18,515::__init__::225::IOProcess::(_processLogs) (1348) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:18,516::__init__::225::IOProcess::(_processLogs) DEBUG|(1348) Building response >Thread-17::DEBUG::2014-08-22 09:46:18,517::__init__::225::IOProcess::(_processLogs) (1348) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:18,517::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:18,517::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:19,348::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:19,348::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:19,348::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:19,348::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:19,348::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:19,348::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:19,349::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:19,349::__init__::225::IOProcess::(_processLogs) (1349) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:19,349::__init__::225::IOProcess::(_processLogs) (1349) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:19,349::__init__::225::IOProcess::(_processLogs) DEBUG|(1349) Building response >Thread-22::DEBUG::2014-08-22 09:46:19,350::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:19,350::__init__::225::IOProcess::(_processLogs) (1349) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:19,355::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:19,356::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:46:19,356::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000210448 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:19,356::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:19,357::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:19,357::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:19,358::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:19,358::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:19,358::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:19,358::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:19,358::__init__::225::IOProcess::(_processLogs) (1350) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:19,359::__init__::225::IOProcess::(_processLogs) (1350) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:19,359::__init__::225::IOProcess::(_processLogs) (1350) Building response >Thread-17::DEBUG::2014-08-22 09:46:19,359::__init__::225::IOProcess::(_processLogs) (1350) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:19,359::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:19,359::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:19,360::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:19,360::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:46:19,360::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:19,360::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) (1351) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) (1351) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) (1351) Building response >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) (1351) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:19,361::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:19,362::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:19,362::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:46:19,362::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:19,362::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:19,362::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:19,362::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) (1352) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) (1352) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) (1352) Building response >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) (1352) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:46:19,363::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:19,364::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:19,364::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:19,364::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:19,364::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:19,365::__init__::225::IOProcess::(_processLogs) (1353) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:19,365::__init__::225::IOProcess::(_processLogs) (1353) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:19,365::__init__::225::IOProcess::(_processLogs) (1353) Building response >Thread-17::DEBUG::2014-08-22 09:46:19,365::__init__::225::IOProcess::(_processLogs) (1353) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:19,365::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:19,365::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:46:19,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:19,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120454 s, 85.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:20,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc726530-1cd8-421f-aa54-1b80776cc814`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:20,222::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:46:20,222::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199868', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000210448', 'lastCheck': '0.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:20,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc726530-1cd8-421f-aa54-1b80776cc814`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199868', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000210448', 'lastCheck': '0.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:20,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc726530-1cd8-421f-aa54-1b80776cc814`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:20,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:20,222::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:20,223::task::993::Storage.TaskManager.Task::(_decref) Task=`bc726530-1cd8-421f-aa54-1b80776cc814`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:21,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:22,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114613 s, 89.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:22,162::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:46:22,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`c05785b2-de0c-43ee-9a9c-71440c830032`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:22,166::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:46:22,166::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:22,167::task::1191::Storage.TaskManager.Task::(prepare) Task=`c05785b2-de0c-43ee-9a9c-71440c830032`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:22,167::task::595::Storage.TaskManager.Task::(_updateState) Task=`c05785b2-de0c-43ee-9a9c-71440c830032`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:22,167::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:22,167::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:22,167::task::993::Storage.TaskManager.Task::(_decref) Task=`c05785b2-de0c-43ee-9a9c-71440c830032`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:46:22,243::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:46:22,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`e23f45e1-f8db-4851-9424-3e795337bc09`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:46:22,243::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:46:22,243::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f50c2a4d-3ed4-45e2-aeec-c1bcb571beaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:46:22,243::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:46:22,244::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:46:22,244::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f50c2a4d-3ed4-45e2-aeec-c1bcb571beaf`::Granted request >Thread-46::DEBUG::2014-08-22 09:46:22,244::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e23f45e1-f8db-4851-9424-3e795337bc09`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:46:22,244::task::993::Storage.TaskManager.Task::(_decref) Task=`e23f45e1-f8db-4851-9424-3e795337bc09`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:46:22,246::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:22,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`e23f45e1-f8db-4851-9424-3e795337bc09`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108754108416', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:22,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`e23f45e1-f8db-4851-9424-3e795337bc09`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:46:22,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:46:22,246::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:46:22,246::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:46:22,247::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:46:22,247::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:46:22,247::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:46:22,247::task::993::Storage.TaskManager.Task::(_decref) Task=`e23f45e1-f8db-4851-9424-3e795337bc09`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:24,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:24,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117097 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:26,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:26,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114563 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:28,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:28,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117833 s, 86.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:28,517::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:28,517::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:28,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:28,518::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:46:28,519::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:28,519::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:28,525::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:28,526::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:46:28,526::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219638 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:28,527::__init__::225::IOProcess::(_processLogs) (1354) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:28,527::__init__::225::IOProcess::(_processLogs) (1354) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:28,527::__init__::225::IOProcess::(_processLogs) (1354) Building response >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) (1354) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:28,528::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:28,529::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:28,529::__init__::225::IOProcess::(_processLogs) (1355) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:28,529::__init__::225::IOProcess::(_processLogs) (1355) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:28,529::__init__::225::IOProcess::(_processLogs) (1355) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:28,530::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:28,530::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) (1356) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:29,365::__init__::225::IOProcess::(_processLogs) (1356) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:29,366::__init__::225::IOProcess::(_processLogs) (1356) Queuing response >Thread-22::DEBUG::2014-08-22 09:46:29,366::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:29,366::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:29,372::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:46:29,373::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000226423 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:29,374::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:29,374::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:29,374::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:29,374::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:29,374::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:29,374::__init__::225::IOProcess::(_processLogs) (1357) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:29,375::__init__::225::IOProcess::(_processLogs) (1357) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:29,375::__init__::225::IOProcess::(_processLogs) (1357) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:29,376::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:29,376::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:29,376::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:46:29,376::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) (1358) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) (1358) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) (1358) Building response >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) (1358) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:29,377::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:29,378::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:29,378::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:46:29,378::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:29,378::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) (1359) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) (1359) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) (1359) Building response >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) (1359) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:29,379::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:46:29,380::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:29,380::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:29,380::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:29,381::__init__::225::IOProcess::(_processLogs) (1360) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:29,381::__init__::225::IOProcess::(_processLogs) (1360) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:29,381::__init__::225::IOProcess::(_processLogs) (1360) Building response >Thread-17::DEBUG::2014-08-22 09:46:29,381::__init__::225::IOProcess::(_processLogs) (1360) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:29,381::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:29,381::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:46:30,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:30,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119889 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:32,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:32,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191742 s, 53.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:32,299::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3e0a2e8c] >Thread-47::DEBUG::2014-08-22 09:46:32,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`1976f428-c635-4f5a-8451-9d09f3d070fe`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:32,302::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:46:32,304::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:32,305::task::1191::Storage.TaskManager.Task::(prepare) Task=`1976f428-c635-4f5a-8451-9d09f3d070fe`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:32,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`1976f428-c635-4f5a-8451-9d09f3d070fe`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:32,307::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:32,307::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:32,308::task::993::Storage.TaskManager.Task::(_decref) Task=`1976f428-c635-4f5a-8451-9d09f3d070fe`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:46:32,334::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3e0a2e8c] >Thread-46::DEBUG::2014-08-22 09:46:32,335::task::595::Storage.TaskManager.Task::(_updateState) Task=`804b65be-6e23-4147-a6b1-e03d59577153`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:46:32,336::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:46:32,339::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5054e7d2-1f6f-4c00-a828-4537afbb6e4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:46:32,339::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:46:32,341::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:46:32,341::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5054e7d2-1f6f-4c00-a828-4537afbb6e4b`::Granted request >Thread-46::DEBUG::2014-08-22 09:46:32,367::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`804b65be-6e23-4147-a6b1-e03d59577153`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:46:32,369::task::993::Storage.TaskManager.Task::(_decref) Task=`804b65be-6e23-4147-a6b1-e03d59577153`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:46:32,389::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108637716480', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108637716480', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:32,390::task::1191::Storage.TaskManager.Task::(prepare) Task=`804b65be-6e23-4147-a6b1-e03d59577153`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108637716480', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108637716480', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:32,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`804b65be-6e23-4147-a6b1-e03d59577153`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:46:32,392::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:46:32,392::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:46:32,395::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:46:32,397::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:46:32,398::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:46:32,398::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:46:32,400::task::993::Storage.TaskManager.Task::(_decref) Task=`804b65be-6e23-4147-a6b1-e03d59577153`::ref 0 aborting False >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,594::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,596::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,598::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,599::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme spice subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,600::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,601::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >libvirtEventLoop::DEBUG::2014-08-22 09:46:32,602::clientIF::602::vm.Vm::(dispatchLibvirtEvents) vmId=`61901e42-5ccc-43da-aaa7-60ed6bfbd5f5`::graphics event phase 2 localAddr {'node': '10.34.62.204', 'service': '', 'family': 0} remoteAddr {'node': '10.34.131.3', 'service': '', 'family': 0}authScheme subject [] >Dummy-45::DEBUG::2014-08-22 09:46:34,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:34,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116886 s, 87.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:35,419::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9b81527-0a91-48e6-b567-8b8db7088602`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:35,420::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:46:35,420::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219638', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000226423', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:35,420::task::1191::Storage.TaskManager.Task::(prepare) Task=`e9b81527-0a91-48e6-b567-8b8db7088602`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219638', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000226423', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:35,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9b81527-0a91-48e6-b567-8b8db7088602`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:35,420::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:35,420::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:35,420::task::993::Storage.TaskManager.Task::(_decref) Task=`e9b81527-0a91-48e6-b567-8b8db7088602`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:36,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:36,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114726 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) (1361) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:38,531::__init__::225::IOProcess::(_processLogs) (1361) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:38,532::__init__::225::IOProcess::(_processLogs) (1361) Queuing response >Thread-21::DEBUG::2014-08-22 09:46:38,533::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:38,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:38,539::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:46:38,540::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000215119 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) (1362) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:38,541::__init__::225::IOProcess::(_processLogs) (1362) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:38,542::__init__::225::IOProcess::(_processLogs) (1362) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:38,543::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:38,543::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:46:38,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:38,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117184 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:39,381::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:39,381::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:39,381::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:39,381::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:39,381::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:39,381::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:39,382::__init__::225::IOProcess::(_processLogs) (1363) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:39,382::__init__::225::IOProcess::(_processLogs) (1363) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:39,382::__init__::225::IOProcess::(_processLogs) (1363) Queuing response >Thread-22::DEBUG::2014-08-22 09:46:39,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:39,383::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:39,389::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:46:39,390::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000149359 s, 3.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:39,390::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:39,390::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:39,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:39,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:39,391::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:39,391::__init__::225::IOProcess::(_processLogs) (1364) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:39,391::__init__::225::IOProcess::(_processLogs) (1364) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:39,392::__init__::225::IOProcess::(_processLogs) (1364) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:39,392::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:39,392::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:39,392::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:46:39,393::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:39,393::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:39,393::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:39,393::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:39,393::__init__::225::IOProcess::(_processLogs) (1365) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) (1365) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) (1365) Building response >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) (1365) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:46:39,394::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) (1366) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) (1366) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) (1366) Building response >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) (1366) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:39,395::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:39,396::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:46:39,396::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:39,396::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:39,396::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) (1367) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) (1367) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) (1367) Building response >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) (1367) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:39,397::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:46:40,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:40,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191056 s, 53.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:42,452::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:46:42,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`09e4bbc5-672b-4832-bffc-7d1f6a20b83b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:42,453::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:46:42,453::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:42,453::task::1191::Storage.TaskManager.Task::(prepare) Task=`09e4bbc5-672b-4832-bffc-7d1f6a20b83b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:42,453::task::595::Storage.TaskManager.Task::(_updateState) Task=`09e4bbc5-672b-4832-bffc-7d1f6a20b83b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:42,453::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:42,453::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:42,453::task::993::Storage.TaskManager.Task::(_decref) Task=`09e4bbc5-672b-4832-bffc-7d1f6a20b83b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:46:42,487::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:46:42,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`97dc4524-8483-41b7-84ac-f1173e198081`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:46:42,487::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:46:42,488::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a9136bb1-3a60-45fb-bc1c-b57f6007733a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:46:42,488::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:46:42,488::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:46:42,488::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a9136bb1-3a60-45fb-bc1c-b57f6007733a`::Granted request >Thread-46::DEBUG::2014-08-22 09:46:42,488::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`97dc4524-8483-41b7-84ac-f1173e198081`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:46:42,488::task::993::Storage.TaskManager.Task::(_decref) Task=`97dc4524-8483-41b7-84ac-f1173e198081`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:46:42,490::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108580044800', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108580044800', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:42,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`97dc4524-8483-41b7-84ac-f1173e198081`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108580044800', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108580044800', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:42,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`97dc4524-8483-41b7-84ac-f1173e198081`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:46:42,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:46:42,491::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:46:42,491::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:46:42,491::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:46:42,491::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:46:42,491::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:46:42,491::task::993::Storage.TaskManager.Task::(_decref) Task=`97dc4524-8483-41b7-84ac-f1173e198081`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:42,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:42,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118116 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:44,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:44,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116951 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:46,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:46,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115684 s, 88.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:48,543::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:48,543::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:48,543::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:48,544::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:48,544::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:48,544::__init__::225::IOProcess::(_processLogs) (1368) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:48,544::__init__::225::IOProcess::(_processLogs) (1368) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:48,544::__init__::225::IOProcess::(_processLogs) (1368) Queuing response >Thread-21::DEBUG::2014-08-22 09:46:48,545::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:48,545::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:48,551::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:46:48,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178023 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) (1369) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:48,553::__init__::225::IOProcess::(_processLogs) (1369) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:48,554::__init__::225::IOProcess::(_processLogs) (1369) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:48,555::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:48,555::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:46:48,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:48,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118649 s, 86.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:49,397::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:49,397::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:49,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:49,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:49,398::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:49,398::__init__::225::IOProcess::(_processLogs) (1370) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:49,398::__init__::225::IOProcess::(_processLogs) (1370) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:49,398::__init__::225::IOProcess::(_processLogs) (1370) Queuing response >Thread-22::DEBUG::2014-08-22 09:46:49,399::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:49,399::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:49,405::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:46:49,406::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000186437 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) (1371) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:49,407::__init__::225::IOProcess::(_processLogs) (1371) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:49,408::__init__::225::IOProcess::(_processLogs) (1371) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:49,408::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:49,409::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:49,409::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:46:49,409::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:49,409::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) (1372) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) (1372) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) (1372) Building response >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) (1372) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:46:49,410::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) (1373) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) (1373) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) (1373) Building response >Thread-17::DEBUG::2014-08-22 09:46:49,411::__init__::225::IOProcess::(_processLogs) (1373) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:49,412::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:49,412::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:49,412::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:46:49,412::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:49,412::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) (1374) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) (1374) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) (1374) Building response >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) (1374) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:49,413::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:46:50,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2d28dce-1e3d-47e5-8315-a61e5a672157`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:50,737::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:46:50,737::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178023', 'lastCheck': '2.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000186437', 'lastCheck': '1.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:50,737::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2d28dce-1e3d-47e5-8315-a61e5a672157`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178023', 'lastCheck': '2.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000186437', 'lastCheck': '1.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:46:50,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2d28dce-1e3d-47e5-8315-a61e5a672157`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:50,738::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:50,738::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:50,738::task::993::Storage.TaskManager.Task::(_decref) Task=`e2d28dce-1e3d-47e5-8315-a61e5a672157`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:50,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:50,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119284 s, 85.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:46:52,561::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:46:52,561::task::595::Storage.TaskManager.Task::(_updateState) Task=`8be63f39-00b1-4eb0-a701-92cc634c0b13`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:46:52,561::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:46:52,562::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:52,562::task::1191::Storage.TaskManager.Task::(prepare) Task=`8be63f39-00b1-4eb0-a701-92cc634c0b13`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:46:52,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`8be63f39-00b1-4eb0-a701-92cc634c0b13`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:46:52,562::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:46:52,562::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:46:52,562::task::993::Storage.TaskManager.Task::(_decref) Task=`8be63f39-00b1-4eb0-a701-92cc634c0b13`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:46:52,600::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:46:52,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`7df2604b-c5c8-4ca7-8baa-69596bd6d7fd`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:46:52,600::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:46:52,600::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`05ef77d2-5fd6-4203-90cd-993c5ef51396`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:46:52,600::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:46:52,601::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:46:52,601::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`05ef77d2-5fd6-4203-90cd-993c5ef51396`::Granted request >Thread-46::DEBUG::2014-08-22 09:46:52,601::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7df2604b-c5c8-4ca7-8baa-69596bd6d7fd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:46:52,601::task::993::Storage.TaskManager.Task::(_decref) Task=`7df2604b-c5c8-4ca7-8baa-69596bd6d7fd`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:46:52,603::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108507693056', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108493012992', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:52,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`7df2604b-c5c8-4ca7-8baa-69596bd6d7fd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108507693056', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108493012992', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:46:52,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`7df2604b-c5c8-4ca7-8baa-69596bd6d7fd`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:46:52,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:46:52,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:46:52,603::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:46:52,603::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:46:52,604::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:46:52,604::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:46:52,604::task::993::Storage.TaskManager.Task::(_decref) Task=`7df2604b-c5c8-4ca7-8baa-69596bd6d7fd`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:46:52,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:52,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116823 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:54,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:55,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115166 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:46:57,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:57,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115442 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:58,555::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) (1375) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) (1375) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:58,556::__init__::225::IOProcess::(_processLogs) (1375) Building response >Thread-17::DEBUG::2014-08-22 09:46:58,557::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:46:58,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:46:58,558::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:46:58,566::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00150404 s, 264 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) (1376) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:58,567::__init__::225::IOProcess::(_processLogs) (1376) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:58,568::__init__::225::IOProcess::(_processLogs) DEBUG|(1376) Building response >Thread-17::DEBUG::2014-08-22 09:46:58,569::__init__::225::IOProcess::(_processLogs) (1376) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:58,569::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:58,569::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:46:59,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:46:59,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218113 s, 46.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:59,413::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) (1377) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:59,414::__init__::225::IOProcess::(_processLogs) (1377) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:59,419::__init__::225::IOProcess::(_processLogs) DEBUG|(1377) Building response >Thread-17::DEBUG::2014-08-22 09:46:59,419::__init__::225::IOProcess::(_processLogs) (1377) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:59,419::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:46:59,420::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:46:59,420::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:46:59,436::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00960113 s, 59.4 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:46:59,436::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) (1378) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:46:59,437::__init__::225::IOProcess::(_processLogs) (1378) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:46:59,438::__init__::225::IOProcess::(_processLogs) (1378) Building response >Thread-17::DEBUG::2014-08-22 09:46:59,438::__init__::225::IOProcess::(_processLogs) (1378) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:59,438::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:59,439::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:46:59,439::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:46:59,439::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:59,439::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) (1379) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) (1379) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) (1379) Building response >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) (1379) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:59,440::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) (1380) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:59,441::__init__::225::IOProcess::(_processLogs) (1380) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:59,469::__init__::225::IOProcess::(_processLogs) (1380) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:59,470::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:46:59,470::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:46:59,470::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:46:59,470::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:46:59,470::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:46:59,470::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:46:59,471::__init__::225::IOProcess::(_processLogs) (1381) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:46:59,471::__init__::225::IOProcess::(_processLogs) (1381) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:46:59,500::__init__::225::IOProcess::(_processLogs) (1381) Queuing response >Thread-17::DEBUG::2014-08-22 09:46:59,500::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:46:59,500::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:47:01,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:01,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121299 s, 84.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:02,653::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:47:02,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`3768d640-a1a3-4ad7-b94b-c9dce8112337`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:02,654::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:47:02,655::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:02,656::task::1191::Storage.TaskManager.Task::(prepare) Task=`3768d640-a1a3-4ad7-b94b-c9dce8112337`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:02,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`3768d640-a1a3-4ad7-b94b-c9dce8112337`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:02,657::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:02,657::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:02,658::task::993::Storage.TaskManager.Task::(_decref) Task=`3768d640-a1a3-4ad7-b94b-c9dce8112337`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:47:02,695::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:47:02,696::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7f71949-93bb-4606-b601-85993c3db952`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:47:02,697::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:47:02,698::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`597beaa6-704e-4f9e-a48e-62c117edafc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:47:02,698::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:47:02,699::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:47:02,700::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`597beaa6-704e-4f9e-a48e-62c117edafc0`::Granted request >Thread-46::DEBUG::2014-08-22 09:47:02,701::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e7f71949-93bb-4606-b601-85993c3db952`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:47:02,701::task::993::Storage.TaskManager.Task::(_decref) Task=`e7f71949-93bb-4606-b601-85993c3db952`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:47:02,711::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108426952704', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108422758400', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:02,711::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7f71949-93bb-4606-b601-85993c3db952`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108426952704', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108422758400', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:02,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7f71949-93bb-4606-b601-85993c3db952`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:47:02,712::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:47:02,712::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:47:02,713::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:47:02,714::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:47:02,714::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:47:02,715::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:47:02,715::task::993::Storage.TaskManager.Task::(_decref) Task=`e7f71949-93bb-4606-b601-85993c3db952`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:03,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:03,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:05,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:05,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114703 s, 89.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:05,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`a91ca628-8b4d-46c8-8695-94bfe5c5d462`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:05,923::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:47:05,923::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00150404', 'lastCheck': '7.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00960113', 'lastCheck': '6.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:05,923::task::1191::Storage.TaskManager.Task::(prepare) Task=`a91ca628-8b4d-46c8-8695-94bfe5c5d462`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00150404', 'lastCheck': '7.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00960113', 'lastCheck': '6.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:05,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`a91ca628-8b4d-46c8-8695-94bfe5c5d462`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:05,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:05,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:05,924::task::993::Storage.TaskManager.Task::(_decref) Task=`a91ca628-8b4d-46c8-8695-94bfe5c5d462`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:07,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:07,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114306 s, 89.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:08,569::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) (1382) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) (1382) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) (1382) Building response >Thread-17::DEBUG::2014-08-22 09:47:08,570::__init__::225::IOProcess::(_processLogs) (1382) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:08,571::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:47:08,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:47:08,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00024728 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:08,578::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:08,578::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:08,579::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:08,579::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:08,579::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:08,579::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:08,579::__init__::225::IOProcess::(_processLogs) (1383) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:08,579::__init__::225::IOProcess::(_processLogs) (1383) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:08,580::__init__::225::IOProcess::(_processLogs) DEBUG|(1383) Building response >Thread-17::DEBUG::2014-08-22 09:47:08,580::__init__::225::IOProcess::(_processLogs) (1383) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:08,580::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:08,581::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:47:09,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:09,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118338 s, 86.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:09,506::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:09,507::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:09,507::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:09,508::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:09,509::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:09,509::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:09,512::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-22::DEBUG::2014-08-22 09:47:09,513::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:09,513::__init__::225::IOProcess::(_processLogs) (1384) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:09,519::__init__::225::IOProcess::(_processLogs) (1384) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:09,521::__init__::225::IOProcess::(_processLogs) (1384) Building response >Thread-22::DEBUG::2014-08-22 09:47:09,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000226973 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:09,522::__init__::225::IOProcess::(_processLogs) (1384) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:09,523::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:09,523::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:09,525::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:09,525::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:09,526::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:09,526::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:09,527::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:09,528::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:09,530::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:09,531::__init__::225::IOProcess::(_processLogs) (1385) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:09,532::__init__::225::IOProcess::(_processLogs) (1385) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:09,532::__init__::225::IOProcess::(_processLogs) (1385) Building response >Thread-17::DEBUG::2014-08-22 09:47:09,533::__init__::225::IOProcess::(_processLogs) (1385) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:09,533::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:09,534::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:09,535::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:09,536::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:47:09,536::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:09,537::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:09,537::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:09,538::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:09,539::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:09,540::__init__::225::IOProcess::(_processLogs) (1386) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:09,541::__init__::225::IOProcess::(_processLogs) (1386) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:09,541::__init__::225::IOProcess::(_processLogs) (1386) Building response >Thread-17::DEBUG::2014-08-22 09:47:09,542::__init__::225::IOProcess::(_processLogs) (1386) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:09,542::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:09,543::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:09,544::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:47:09,545::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:09,545::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:09,546::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:09,547::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:09,548::__init__::225::IOProcess::(_processLogs) (1387) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:09,549::__init__::225::IOProcess::(_processLogs) (1387) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:09,549::__init__::225::IOProcess::(_processLogs) (1387) Building response >Thread-17::DEBUG::2014-08-22 09:47:09,550::__init__::225::IOProcess::(_processLogs) (1387) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:09,550::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:09,551::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:09,552::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:47:09,553::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:09,553::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:09,554::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:09,554::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:09,555::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:09,556::__init__::225::IOProcess::(_processLogs) (1388) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:09,557::__init__::225::IOProcess::(_processLogs) (1388) Building response >Thread-17::DEBUG::2014-08-22 09:47:09,558::__init__::225::IOProcess::(_processLogs) (1388) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:09,558::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:09,559::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:47:11,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:11,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119227 s, 85.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:12,818::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:47:12,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a7f322d-3386-44ee-8bea-54a207e89fb6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:12,822::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:47:12,822::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:12,822::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a7f322d-3386-44ee-8bea-54a207e89fb6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:12,822::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a7f322d-3386-44ee-8bea-54a207e89fb6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:12,822::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:12,822::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:12,823::task::993::Storage.TaskManager.Task::(_decref) Task=`2a7f322d-3386-44ee-8bea-54a207e89fb6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:47:12,857::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:47:12,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a7ea3a0-e89b-4334-be17-cab38939f260`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:47:12,857::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:47:12,858::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b56bf406-c472-4ceb-9bb6-40715ffd54dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:47:12,858::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:47:12,858::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:47:12,858::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b56bf406-c472-4ceb-9bb6-40715ffd54dd`::Granted request >Thread-46::DEBUG::2014-08-22 09:47:12,858::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5a7ea3a0-e89b-4334-be17-cab38939f260`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:47:12,858::task::993::Storage.TaskManager.Task::(_decref) Task=`5a7ea3a0-e89b-4334-be17-cab38939f260`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:47:12,860::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108311609344', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108311609344', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:12,860::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a7ea3a0-e89b-4334-be17-cab38939f260`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108311609344', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108311609344', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:12,861::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a7ea3a0-e89b-4334-be17-cab38939f260`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:47:12,861::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:47:12,861::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:47:12,861::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:47:12,861::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:47:12,861::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:47:12,861::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:47:12,861::task::993::Storage.TaskManager.Task::(_decref) Task=`5a7ea3a0-e89b-4334-be17-cab38939f260`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:13,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:13,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116253 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:15,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:15,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117009 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:17,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:17,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114374 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:18,581::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:18,581::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:18,581::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:18,581::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:18,581::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:18,581::__init__::225::IOProcess::(_processLogs) (1389) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:18,582::__init__::225::IOProcess::(_processLogs) (1389) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:18,582::__init__::225::IOProcess::(_processLogs) (1389) Queuing response >Thread-21::DEBUG::2014-08-22 09:47:18,583::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:18,583::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:18,589::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:47:18,590::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000207109 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) (1390) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:18,591::__init__::225::IOProcess::(_processLogs) (1390) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:18,592::__init__::225::IOProcess::(_processLogs) (1390) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:18,593::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:18,593::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:19,557::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:19,557::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:19,557::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:19,557::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:19,558::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:19,558::__init__::225::IOProcess::(_processLogs) (1391) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:19,558::__init__::225::IOProcess::(_processLogs) (1391) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:19,558::__init__::225::IOProcess::(_processLogs) (1391) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:19,558::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:47:19,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:47:19,565::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00025688 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:19,565::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:19,565::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:19,566::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:19,566::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:19,566::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:19,566::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:19,566::__init__::225::IOProcess::(_processLogs) (1392) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:19,566::__init__::225::IOProcess::(_processLogs) (1392) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:19,567::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1392) Building response >Thread-17::DEBUG::2014-08-22 09:47:19,567::__init__::225::IOProcess::(_processLogs) (1392) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:19,567::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) (1393) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:19,568::__init__::225::IOProcess::(_processLogs) (1393) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:19,569::__init__::225::IOProcess::(_processLogs) (1393) Building response >Thread-17::DEBUG::2014-08-22 09:47:19,569::__init__::225::IOProcess::(_processLogs) (1393) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:19,569::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:47:19,569::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:19,569::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:19,570::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:47:19,570::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:19,570::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:19,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:19,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:19,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) (1394) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) (1394) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) (1394) Building response >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) (1394) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:47:19,571::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:19,572::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:19,572::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:19,572::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:19,572::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:19,573::__init__::225::IOProcess::(_processLogs) (1395) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:19,573::__init__::225::IOProcess::(_processLogs) (1395) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:19,573::__init__::225::IOProcess::(_processLogs) (1395) Building response >Thread-17::DEBUG::2014-08-22 09:47:19,573::__init__::225::IOProcess::(_processLogs) (1395) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:19,573::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:19,573::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:47:19,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:19,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011798 s, 86.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:21,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1a6d5c8-d66b-4125-8585-06c536afeae8`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:21,221::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:47:21,221::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000207109', 'lastCheck': '2.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00025688', 'lastCheck': '1.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:21,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1a6d5c8-d66b-4125-8585-06c536afeae8`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000207109', 'lastCheck': '2.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00025688', 'lastCheck': '1.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:21,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1a6d5c8-d66b-4125-8585-06c536afeae8`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:21,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:21,222::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:21,222::task::993::Storage.TaskManager.Task::(_decref) Task=`d1a6d5c8-d66b-4125-8585-06c536afeae8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:21,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:21,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116318 s, 88.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:22,934::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:47:22,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`77f5b858-6a06-43c8-9801-c173ce25b337`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:22,935::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:47:22,935::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:22,935::task::1191::Storage.TaskManager.Task::(prepare) Task=`77f5b858-6a06-43c8-9801-c173ce25b337`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:22,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`77f5b858-6a06-43c8-9801-c173ce25b337`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:22,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:22,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:22,936::task::993::Storage.TaskManager.Task::(_decref) Task=`77f5b858-6a06-43c8-9801-c173ce25b337`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:47:23,011::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:47:23,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac10fe46-37e9-416b-af5f-7b3406128223`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:47:23,012::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:47:23,012::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`df621fe8-030a-4e9d-99e7-13e5edee0e0d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:47:23,012::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:47:23,012::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:47:23,012::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`df621fe8-030a-4e9d-99e7-13e5edee0e0d`::Granted request >Thread-46::DEBUG::2014-08-22 09:47:23,013::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ac10fe46-37e9-416b-af5f-7b3406128223`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:47:23,013::task::993::Storage.TaskManager.Task::(_decref) Task=`ac10fe46-37e9-416b-af5f-7b3406128223`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:47:23,015::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108157468672', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108157468672', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:23,015::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac10fe46-37e9-416b-af5f-7b3406128223`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108157468672', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108157468672', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:23,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac10fe46-37e9-416b-af5f-7b3406128223`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:47:23,015::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:47:23,015::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:47:23,015::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:47:23,015::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:47:23,015::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:47:23,015::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:47:23,016::task::993::Storage.TaskManager.Task::(_decref) Task=`ac10fe46-37e9-416b-af5f-7b3406128223`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:23,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:23,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117241 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:25,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:25,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117119 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:27,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:28,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114936 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:28,593::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:28,593::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:28,594::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:28,594::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:47:28,595::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:28,595::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:28,601::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:28,601::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:28,601::__init__::225::IOProcess::(_processLogs) (1396) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:28,602::__init__::225::IOProcess::(_processLogs) (1396) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:28,602::__init__::225::IOProcess::(_processLogs) (1396) Building response >Thread-17::DEBUG::2014-08-22 09:47:28,602::__init__::225::IOProcess::(_processLogs) (1396) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:28,602::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:28,602::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:47:28,662::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.025837 s, 15.4 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:28,662::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:28,662::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) (1397) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:28,663::__init__::225::IOProcess::(_processLogs) (1397) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:28,664::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1397) Building response >Thread-17::DEBUG::2014-08-22 09:47:28,665::__init__::225::IOProcess::(_processLogs) (1397) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:28,665::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:28,665::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) (1398) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:29,573::__init__::225::IOProcess::(_processLogs) (1398) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:29,574::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1398) Building response >Thread-22::DEBUG::2014-08-22 09:47:29,575::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:29,575::__init__::225::IOProcess::(_processLogs) (1398) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:29,581::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:29,581::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:47:29,582::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000321677 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:29,582::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) (1399) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:29,583::__init__::225::IOProcess::(_processLogs) (1399) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:29,584::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1399) Building response >Thread-17::DEBUG::2014-08-22 09:47:29,585::__init__::225::IOProcess::(_processLogs) (1399) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:29,585::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:29,585::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:29,585::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:29,585::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:47:29,585::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) (1400) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) (1400) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) (1400) Building response >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) (1400) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:29,586::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:29,587::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:29,587::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:29,587::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:47:29,587::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:29,587::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) (1401) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) (1401) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) (1401) Building response >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) (1401) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:29,588::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:29,589::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:47:29,589::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:29,589::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:29,589::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) (1402) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) (1402) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) (1402) Building response >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) (1402) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:29,590::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:47:30,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:30,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020468 s, 50.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:32,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:32,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214987 s, 47.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:33,062::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:47:33,062::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9de5dc4-8b79-45c9-b336-4a7b2896e490`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:33,062::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:47:33,063::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:33,063::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9de5dc4-8b79-45c9-b336-4a7b2896e490`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:33,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9de5dc4-8b79-45c9-b336-4a7b2896e490`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:33,063::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:33,063::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:33,063::task::993::Storage.TaskManager.Task::(_decref) Task=`c9de5dc4-8b79-45c9-b336-4a7b2896e490`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:47:33,087::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:47:33,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`232fa6c7-c349-4ef6-8602-995db3eb5a5a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:47:33,088::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:47:33,088::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6776c071-8c36-4f92-9e96-d64f301ec60e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:47:33,088::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:47:33,088::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:47:33,088::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6776c071-8c36-4f92-9e96-d64f301ec60e`::Granted request >Thread-46::DEBUG::2014-08-22 09:47:33,088::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`232fa6c7-c349-4ef6-8602-995db3eb5a5a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:47:33,089::task::993::Storage.TaskManager.Task::(_decref) Task=`232fa6c7-c349-4ef6-8602-995db3eb5a5a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:47:33,090::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108107137024', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108058902528', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:33,091::task::1191::Storage.TaskManager.Task::(prepare) Task=`232fa6c7-c349-4ef6-8602-995db3eb5a5a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '108107137024', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '108058902528', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:33,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`232fa6c7-c349-4ef6-8602-995db3eb5a5a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:47:33,091::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:47:33,091::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:47:33,091::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:47:33,091::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:47:33,091::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:47:33,091::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:47:33,091::task::993::Storage.TaskManager.Task::(_decref) Task=`232fa6c7-c349-4ef6-8602-995db3eb5a5a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:34,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:34,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117608 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:36,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:36,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114117 s, 89.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:36,436::task::595::Storage.TaskManager.Task::(_updateState) Task=`a23305b4-0711-4772-8d93-ec29b8634f6f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:36,436::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:47:36,436::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.025837', 'lastCheck': '7.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000321677', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:36,437::task::1191::Storage.TaskManager.Task::(prepare) Task=`a23305b4-0711-4772-8d93-ec29b8634f6f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.025837', 'lastCheck': '7.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000321677', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:36,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`a23305b4-0711-4772-8d93-ec29b8634f6f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:36,437::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:36,437::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:36,437::task::993::Storage.TaskManager.Task::(_decref) Task=`a23305b4-0711-4772-8d93-ec29b8634f6f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:38,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:38,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116941 s, 87.6 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:47:38,665::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:38,667::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,668::__init__::225::IOProcess::(_processLogs) (1403) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:47:38,669::__init__::225::IOProcess::(_processLogs) (1403) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:47:38,670::__init__::225::IOProcess::(_processLogs) (1403) Building response >Thread-17::DEBUG::2014-08-22 09:47:38,671::__init__::225::IOProcess::(_processLogs) (1403) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,672::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,673::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:47:38,674::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,675::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:47:38,676::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,677::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,678::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,679::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:47:38,680::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:47:38,680::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,681::__init__::225::IOProcess::(_processLogs) (1404) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:47:38,681::__init__::225::IOProcess::(_processLogs) (1404) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:47:38,681::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1404) Building response >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) (1404) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,682::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,683::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,683::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:38,683::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,683::__init__::225::IOProcess::(_processLogs) (1405) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:47:38,683::__init__::225::IOProcess::(_processLogs) (1405) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:47:38,732::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1405) Building response >Thread-17::DEBUG::2014-08-22 09:47:38,732::__init__::225::IOProcess::(_processLogs) (1405) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,732::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,732::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 74 >Thread-21::DEBUG::2014-08-22 09:47:38,732::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:47:38,733::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,733::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) (1406) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) (1406) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:47:38,734::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1406) Building response >Thread-21::DEBUG::2014-08-22 09:47:38,735::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:47:38,735::__init__::225::IOProcess::(_processLogs) (1406) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,736::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,736::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 09:47:38,737::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,737::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 09:47:38,737::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,737::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,737::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) (1407) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) (1407) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) (1407) Building response >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) (1407) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,738::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) (1408) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) (1408) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:47:38,739::__init__::225::IOProcess::(_processLogs) DEBUG|(1408) Building response >Thread-21::DEBUG::2014-08-22 09:47:38,739::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-21::INFO::2014-08-22 09:47:38,740::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:47:38,740::__init__::225::IOProcess::(_processLogs) (1408) Queuing response >Thread-21::INFO::2014-08-22 09:47:38,741::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:47:38,741::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,741::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 09:47:38,742::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,742::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:38,742::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,742::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:47:38,743::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:38,743::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:47:38,750::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196643 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:38,750::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) (1409) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) (1409) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) (1409) Building response >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) (1409) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:38,752::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:38,753::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:38,753::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:38,753::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:38,753::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:38,753::__init__::225::IOProcess::(_processLogs) (1410) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:38,753::__init__::225::IOProcess::(_processLogs) (1410) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:38,754::__init__::225::IOProcess::(_processLogs) DEBUG|(1410) Building response >Thread-17::DEBUG::2014-08-22 09:47:38,754::__init__::225::IOProcess::(_processLogs) (1410) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:38,754::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:38,755::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:47:39,590::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:47:39,591::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:39,591::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:47:39,591::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,592::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:39,592::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,592::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,592::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,592::__init__::225::IOProcess::(_processLogs) (1411) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:47:39,592::__init__::225::IOProcess::(_processLogs) (1411) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) (1411) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) (1411) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,593::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:39,594::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,595::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,595::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,596::__init__::225::IOProcess::(_processLogs) (1412) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:47:39,596::__init__::225::IOProcess::(_processLogs) (1412) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:47:39,597::__init__::225::IOProcess::(_processLogs) (1412) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,598::__init__::225::IOProcess::(_processLogs) (1412) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,599::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 09:47:39,600::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:47:39,600::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:47:39,600::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) (1413) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:47:39,601::__init__::225::IOProcess::(_processLogs) (1413) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:47:39,627::__init__::225::IOProcess::(_processLogs) DEBUG|(1413) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,627::__init__::225::IOProcess::(_processLogs) (1413) Queuing response >Thread-22::DEBUG::2014-08-22 09:47:39,627::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:47:39,628::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,628::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:39,628::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 09:47:39,628::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,629::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,629::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,629::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,629::__init__::225::IOProcess::(_processLogs) (1414) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:47:39,629::__init__::225::IOProcess::(_processLogs) (1414) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:47:39,629::__init__::225::IOProcess::(_processLogs) (1414) Queuing response >Thread-22::DEBUG::2014-08-22 09:47:39,629::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:47:39,630::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,630::__init__::225::IOProcess::(_processLogs) Sending response sized 832 >Thread-17::DEBUG::2014-08-22 09:47:39,631::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 09:47:39,631::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,631::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) (1415) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) (1415) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) (1415) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) (1415) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:47:39,632::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,633::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,633::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,633::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,633::__init__::225::IOProcess::(_processLogs) (1416) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:47:39,633::__init__::225::IOProcess::(_processLogs) (1416) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:47:39,633::__init__::225::IOProcess::(_processLogs) (1416) Queuing response >Thread-22::DEBUG::2014-08-22 09:47:39,633::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:47:39,634::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 09:47:39,634::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:47:39,634::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-22::INFO::2014-08-22 09:47:39,634::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) (1417) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:39,635::__init__::225::IOProcess::(_processLogs) (1417) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:39,636::__init__::225::IOProcess::(_processLogs) (1417) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,636::__init__::225::IOProcess::(_processLogs) (1417) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,636::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:47:39,637::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:47:39,643::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000250112 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) (1418) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:39,644::__init__::225::IOProcess::(_processLogs) (1418) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:39,645::__init__::225::IOProcess::(_processLogs) (1418) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,646::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,646::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:39,646::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:47:39,646::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) (1419) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) (1419) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) (1419) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) (1419) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,647::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:39,648::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:47:39,648::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,648::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,648::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,648::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,648::__init__::225::IOProcess::(_processLogs) (1420) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) (1420) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) (1420) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) (1420) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:47:39,649::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:39,650::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:39,650::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:39,650::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:39,651::__init__::225::IOProcess::(_processLogs) (1421) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:39,651::__init__::225::IOProcess::(_processLogs) (1421) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:39,651::__init__::225::IOProcess::(_processLogs) (1421) Building response >Thread-17::DEBUG::2014-08-22 09:47:39,651::__init__::225::IOProcess::(_processLogs) (1421) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:39,651::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:39,651::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:47:40,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:40,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119789 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:42,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:42,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121852 s, 84.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:43,158::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:47:43,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`58f63cc1-8623-4bc0-a6e1-20f1ae46c62d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:43,159::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:47:43,159::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:43,159::task::1191::Storage.TaskManager.Task::(prepare) Task=`58f63cc1-8623-4bc0-a6e1-20f1ae46c62d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:43,159::task::595::Storage.TaskManager.Task::(_updateState) Task=`58f63cc1-8623-4bc0-a6e1-20f1ae46c62d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:43,159::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:43,159::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:43,159::task::993::Storage.TaskManager.Task::(_decref) Task=`58f63cc1-8623-4bc0-a6e1-20f1ae46c62d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:47:43,246::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:47:43,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`3eebce28-2332-49c9-9378-1a7cc9d42c7d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:47:43,247::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:47:43,247::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f2940217-d904-4d60-bcda-afec285e3169`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:47:43,247::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:47:43,247::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:47:43,247::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f2940217-d904-4d60-bcda-afec285e3169`::Granted request >Thread-46::DEBUG::2014-08-22 09:47:43,247::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3eebce28-2332-49c9-9378-1a7cc9d42c7d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:47:43,247::task::993::Storage.TaskManager.Task::(_decref) Task=`3eebce28-2332-49c9-9378-1a7cc9d42c7d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:47:43,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107989696512', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107989696512', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:43,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`3eebce28-2332-49c9-9378-1a7cc9d42c7d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107989696512', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107989696512', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:43,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`3eebce28-2332-49c9-9378-1a7cc9d42c7d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:47:43,250::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:47:43,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:47:43,250::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:47:43,251::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:47:43,251::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:47:43,251::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:47:43,251::task::993::Storage.TaskManager.Task::(_decref) Task=`3eebce28-2332-49c9-9378-1a7cc9d42c7d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:44,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:44,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115037 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:46,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:46,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121178 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:48,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:48,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116536 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) (1422) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:48,755::__init__::225::IOProcess::(_processLogs) (1422) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:48,756::__init__::225::IOProcess::(_processLogs) (1422) Queuing response >Thread-21::DEBUG::2014-08-22 09:47:48,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:48,757::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:48,763::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:47:48,764::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000207929 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:48,764::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:48,765::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:48,765::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:48,765::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:48,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:48,765::__init__::225::IOProcess::(_processLogs) (1423) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:48,765::__init__::225::IOProcess::(_processLogs) (1423) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:48,766::__init__::225::IOProcess::(_processLogs) (1423) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:48,767::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:48,767::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:49,651::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:49,651::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:49,652::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:47:49,652::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:49,653::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:49,659::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:47:49,660::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000165375 s, 3.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:49,660::__init__::225::IOProcess::(_processLogs) (1424) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:49,661::__init__::225::IOProcess::(_processLogs) (1424) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:49,661::__init__::225::IOProcess::(_processLogs) (1424) Building response >Thread-17::DEBUG::2014-08-22 09:47:49,661::__init__::225::IOProcess::(_processLogs) (1424) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:49,661::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:49,661::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) (1425) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:49,662::__init__::225::IOProcess::(_processLogs) (1425) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:49,663::__init__::225::IOProcess::(_processLogs) (1425) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:49,663::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:49,664::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:49,664::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:47:49,664::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:49,664::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) (1426) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) (1426) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) (1426) Building response >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) (1426) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:49,665::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:47:49,666::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:49,666::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:49,666::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:49,666::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:49,666::__init__::225::IOProcess::(_processLogs) (1427) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:49,666::__init__::225::IOProcess::(_processLogs) (1427) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:49,667::__init__::225::IOProcess::(_processLogs) (1427) Building response >Thread-17::DEBUG::2014-08-22 09:47:49,667::__init__::225::IOProcess::(_processLogs) (1427) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:49,667::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:49,667::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:49,667::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:47:49,667::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:49,668::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:49,668::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:49,668::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:49,668::__init__::225::IOProcess::(_processLogs) (1428) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:49,668::__init__::225::IOProcess::(_processLogs) (1428) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:49,668::__init__::225::IOProcess::(_processLogs) (1428) Building response >Thread-17::DEBUG::2014-08-22 09:47:49,669::__init__::225::IOProcess::(_processLogs) (1428) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:49,669::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:49,669::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:47:50,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:50,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012305 s, 83.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:51,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`950fdf7e-dd8e-4b95-8257-891f4bca3463`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:51,957::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:47:51,957::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000207929', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000165375', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:51,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`950fdf7e-dd8e-4b95-8257-891f4bca3463`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000207929', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000165375', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:47:51,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`950fdf7e-dd8e-4b95-8257-891f4bca3463`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:51,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:51,957::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:51,957::task::993::Storage.TaskManager.Task::(_decref) Task=`950fdf7e-dd8e-4b95-8257-891f4bca3463`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:52,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:52,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120385 s, 85.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:47:53,295::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:47:53,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f0a7c3f-405d-4853-8182-f3861ccc1e46`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:47:53,296::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:47:53,296::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:53,296::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f0a7c3f-405d-4853-8182-f3861ccc1e46`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:47:53,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f0a7c3f-405d-4853-8182-f3861ccc1e46`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:47:53,296::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:47:53,296::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:47:53,296::task::993::Storage.TaskManager.Task::(_decref) Task=`4f0a7c3f-405d-4853-8182-f3861ccc1e46`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:47:53,353::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:47:53,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bb8c7de-fad6-4385-9460-1957ecf9cb84`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:47:53,353::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:47:53,353::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4e0aa414-d833-4f75-9cd0-1a977f9e24ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:47:53,353::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:47:53,354::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:47:53,354::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4e0aa414-d833-4f75-9cd0-1a977f9e24ff`::Granted request >Thread-46::DEBUG::2014-08-22 09:47:53,354::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8bb8c7de-fad6-4385-9460-1957ecf9cb84`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:47:53,355::task::993::Storage.TaskManager.Task::(_decref) Task=`8bb8c7de-fad6-4385-9460-1957ecf9cb84`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:47:53,356::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107859673088', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107838701568', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:53,357::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bb8c7de-fad6-4385-9460-1957ecf9cb84`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107859673088', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107838701568', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:47:53,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bb8c7de-fad6-4385-9460-1957ecf9cb84`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:47:53,357::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:47:53,357::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:47:53,357::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:47:53,357::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:47:53,357::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:47:53,357::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:47:53,357::task::993::Storage.TaskManager.Task::(_decref) Task=`8bb8c7de-fad6-4385-9460-1957ecf9cb84`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:47:54,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:54,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030181 s, 33.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:47:56,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:56,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117393 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:58,767::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:58,767::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:58,767::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:58,767::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:58,767::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:58,768::__init__::225::IOProcess::(_processLogs) (1429) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:58,768::__init__::225::IOProcess::(_processLogs) (1429) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:58,768::__init__::225::IOProcess::(_processLogs) (1429) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:58,769::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:47:58,769::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:47:58,775::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000218777 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:58,776::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:47:58,776::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:58,776::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:58,776::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:58,776::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:58,776::__init__::225::IOProcess::(_processLogs) (1430) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:58,777::__init__::225::IOProcess::(_processLogs) (1430) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:58,777::__init__::225::IOProcess::(_processLogs) (1430) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:58,778::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:58,778::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:47:58,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:47:58,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036441 s, 28.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) (1431) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:59,669::__init__::225::IOProcess::(_processLogs) (1431) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:59,670::__init__::225::IOProcess::(_processLogs) (1431) Queuing response >Thread-22::DEBUG::2014-08-22 09:47:59,670::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:47:59,671::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:59,677::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:47:59,678::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000201491 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:47:59,678::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:47:59,678::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:47:59,679::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:59,679::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:59,679::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:59,679::__init__::225::IOProcess::(_processLogs) (1432) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:47:59,679::__init__::225::IOProcess::(_processLogs) (1432) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:47:59,680::__init__::225::IOProcess::(_processLogs) (1432) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:59,680::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:59,680::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:47:59,680::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:47:59,681::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:59,681::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:59,681::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:59,681::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:59,681::__init__::225::IOProcess::(_processLogs) (1433) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:59,681::__init__::225::IOProcess::(_processLogs) (1433) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) (1433) Building response >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) (1433) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:59,682::__init__::225::IOProcess::(_processLogs) (1434) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:59,683::__init__::225::IOProcess::(_processLogs) (1434) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:59,683::__init__::225::IOProcess::(_processLogs) (1434) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:59,683::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:47:59,683::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:47:59,683::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) (1435) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) (1435) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) (1435) Building response >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) (1435) Queuing response >Thread-17::DEBUG::2014-08-22 09:47:59,684::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:48:00,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:01,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121251 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:03,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:03,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011611 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:03,424::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:48:03,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`dff45f24-a3dc-4848-9eb5-7a3034a4f2ca`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:03,424::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:48:03,425::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:03,425::task::1191::Storage.TaskManager.Task::(prepare) Task=`dff45f24-a3dc-4848-9eb5-7a3034a4f2ca`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:03,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`dff45f24-a3dc-4848-9eb5-7a3034a4f2ca`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:03,425::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:03,425::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:03,425::task::993::Storage.TaskManager.Task::(_decref) Task=`dff45f24-a3dc-4848-9eb5-7a3034a4f2ca`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:48:03,466::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:48:03,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`120684e6-b3ed-4414-8f3e-1dd936d425c9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:48:03,466::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:48:03,467::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`62d88123-1410-4273-9131-399b22110e35`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:48:03,467::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:48:03,467::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:48:03,467::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`62d88123-1410-4273-9131-399b22110e35`::Granted request >Thread-46::DEBUG::2014-08-22 09:48:03,467::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`120684e6-b3ed-4414-8f3e-1dd936d425c9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:48:03,467::task::993::Storage.TaskManager.Task::(_decref) Task=`120684e6-b3ed-4414-8f3e-1dd936d425c9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:48:03,469::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107770544128', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107769495552', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:03,469::task::1191::Storage.TaskManager.Task::(prepare) Task=`120684e6-b3ed-4414-8f3e-1dd936d425c9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107770544128', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107769495552', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:03,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`120684e6-b3ed-4414-8f3e-1dd936d425c9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:48:03,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:48:03,470::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:48:03,470::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:48:03,470::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:48:03,470::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:48:03,470::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:48:03,470::task::993::Storage.TaskManager.Task::(_decref) Task=`120684e6-b3ed-4414-8f3e-1dd936d425c9`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:05,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:05,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137558 s, 74.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:07,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`51141711-35db-4e18-ba2f-fefcbd0b2722`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:07,139::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:48:07,139::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000218777', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000201491', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:07,139::task::1191::Storage.TaskManager.Task::(prepare) Task=`51141711-35db-4e18-ba2f-fefcbd0b2722`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000218777', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000201491', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:07,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`51141711-35db-4e18-ba2f-fefcbd0b2722`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:07,139::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:07,139::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:07,140::task::993::Storage.TaskManager.Task::(_decref) Task=`51141711-35db-4e18-ba2f-fefcbd0b2722`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:07,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:07,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115596 s, 88.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:08,778::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:08,779::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:08,779::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:08,779::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:08,779::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:08,779::__init__::225::IOProcess::(_processLogs) (1436) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:08,779::__init__::225::IOProcess::(_processLogs) (1436) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:08,780::__init__::225::IOProcess::(_processLogs) (1436) Queuing response >Thread-21::DEBUG::2014-08-22 09:48:08,780::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:08,781::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:08,787::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:48:08,788::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000208073 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:08,788::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:08,789::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:08,789::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:08,789::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:08,789::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:08,789::__init__::225::IOProcess::(_processLogs) (1437) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:08,789::__init__::225::IOProcess::(_processLogs) (1437) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:08,790::__init__::225::IOProcess::(_processLogs) (1437) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:08,791::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:08,791::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:48:09,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:09,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116518 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:09,685::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:09,685::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:09,685::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:09,686::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:09,686::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:09,686::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:09,686::__init__::225::IOProcess::(_processLogs) (1438) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:09,686::__init__::225::IOProcess::(_processLogs) (1438) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:09,687::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1438) Building response >Thread-22::DEBUG::2014-08-22 09:48:09,687::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:09,687::__init__::225::IOProcess::(_processLogs) (1438) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:09,694::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:09,695::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:48:09,695::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198167 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) (1439) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:09,696::__init__::225::IOProcess::(_processLogs) (1439) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:09,697::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1439) Building response >Thread-17::DEBUG::2014-08-22 09:48:09,698::__init__::225::IOProcess::(_processLogs) (1439) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:09,698::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:09,698::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:09,698::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:09,698::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:48:09,698::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) (1440) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) (1440) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) (1440) Building response >Thread-17::DEBUG::2014-08-22 09:48:09,699::__init__::225::IOProcess::(_processLogs) (1440) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:09,700::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:09,700::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:09,700::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:09,700::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:48:09,700::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:09,700::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) (1441) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) (1441) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) (1441) Building response >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) (1441) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:09,701::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:09,702::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:48:09,702::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:09,702::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:09,702::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) (1442) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) (1442) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) (1442) Building response >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) (1442) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:09,703::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:48:11,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:11,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337728 s, 30.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:13,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:13,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117149 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:13,519::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:48:13,519::task::595::Storage.TaskManager.Task::(_updateState) Task=`04359cce-1b63-4df6-a29b-4892f0100f01`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:13,519::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:48:13,520::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:13,520::task::1191::Storage.TaskManager.Task::(prepare) Task=`04359cce-1b63-4df6-a29b-4892f0100f01`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:13,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`04359cce-1b63-4df6-a29b-4892f0100f01`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:13,520::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:13,520::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:13,520::task::993::Storage.TaskManager.Task::(_decref) Task=`04359cce-1b63-4df6-a29b-4892f0100f01`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:48:13,638::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:48:13,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1415cc1-e4df-40b6-81df-d9c00a10f85f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:48:13,638::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:48:13,638::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0be59229-0efe-4885-aad2-618a727ac817`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:48:13,638::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:48:13,639::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:48:13,639::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0be59229-0efe-4885-aad2-618a727ac817`::Granted request >Thread-46::DEBUG::2014-08-22 09:48:13,639::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a1415cc1-e4df-40b6-81df-d9c00a10f85f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:48:13,639::task::993::Storage.TaskManager.Task::(_decref) Task=`a1415cc1-e4df-40b6-81df-d9c00a10f85f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:48:13,641::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107746426880', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107746426880', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:13,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1415cc1-e4df-40b6-81df-d9c00a10f85f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107746426880', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107746426880', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:13,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1415cc1-e4df-40b6-81df-d9c00a10f85f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:48:13,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:48:13,641::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:48:13,642::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:48:13,642::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:48:13,642::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:48:13,642::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:48:13,642::task::993::Storage.TaskManager.Task::(_decref) Task=`a1415cc1-e4df-40b6-81df-d9c00a10f85f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:15,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:15,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113671 s, 90.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:17,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:17,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116529 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:18,791::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:18,791::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:18,791::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:18,792::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:48:18,793::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:18,793::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:18,799::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:48:18,800::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000149581 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:18,800::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:18,801::__init__::225::IOProcess::(_processLogs) (1443) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:18,801::__init__::225::IOProcess::(_processLogs) (1443) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:18,801::__init__::225::IOProcess::(_processLogs) (1443) Building response >Thread-17::DEBUG::2014-08-22 09:48:18,801::__init__::225::IOProcess::(_processLogs) (1443) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:18,801::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:18,801::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) (1444) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:18,802::__init__::225::IOProcess::(_processLogs) (1444) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:18,803::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1444) Building response >Thread-17::DEBUG::2014-08-22 09:48:18,804::__init__::225::IOProcess::(_processLogs) (1444) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:18,804::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:18,804::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:48:19,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:19,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155226 s, 66.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:19,703::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:19,703::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:19,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:19,703::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:19,704::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:19,704::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:19,704::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:19,704::__init__::225::IOProcess::(_processLogs) (1445) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:19,704::__init__::225::IOProcess::(_processLogs) (1445) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:19,704::__init__::225::IOProcess::(_processLogs) (1445) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:19,705::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:48:19,705::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:19,705::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:48:19,712::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000262625 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:19,712::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:19,712::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:19,713::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:19,713::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:19,713::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:19,713::__init__::225::IOProcess::(_processLogs) (1446) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:19,713::__init__::225::IOProcess::(_processLogs) (1446) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:19,714::__init__::225::IOProcess::(_processLogs) (1446) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:19,714::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:19,714::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:19,714::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) (1447) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) (1447) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:19,715::__init__::225::IOProcess::(_processLogs) (1447) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:19,716::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:19,716::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:19,716::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:48:19,716::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:19,716::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) (1448) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) (1448) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) (1448) Building response >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) (1448) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:19,717::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:48:19,718::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:19,718::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:19,718::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) (1449) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) (1449) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) (1449) Building response >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) (1449) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:19,719::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:48:21,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:21,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118878 s, 86.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:22,327::task::595::Storage.TaskManager.Task::(_updateState) Task=`1306a1d8-15c1-4f31-8212-a50ef9d59fa7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:22,327::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:48:22,328::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000149581', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000262625', 'lastCheck': '2.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:22,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`1306a1d8-15c1-4f31-8212-a50ef9d59fa7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000149581', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000262625', 'lastCheck': '2.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:22,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`1306a1d8-15c1-4f31-8212-a50ef9d59fa7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:22,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:22,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:22,328::task::993::Storage.TaskManager.Task::(_decref) Task=`1306a1d8-15c1-4f31-8212-a50ef9d59fa7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:23,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:23,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115226 s, 88.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:23,814::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:48:23,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8f3b3ad-c558-4cc2-9097-31bfa0112953`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:23,815::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:48:23,815::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:23,815::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8f3b3ad-c558-4cc2-9097-31bfa0112953`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:23,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8f3b3ad-c558-4cc2-9097-31bfa0112953`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:23,815::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:23,815::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:23,816::task::993::Storage.TaskManager.Task::(_decref) Task=`e8f3b3ad-c558-4cc2-9097-31bfa0112953`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:48:23,858::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:48:23,858::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc37771f-fa57-4d45-aa3e-932d3b4c7276`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:48:23,858::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:48:23,858::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4a9b8cf6-f567-48fc-a617-e4df29bb1964`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:48:23,858::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:48:23,858::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:48:23,858::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4a9b8cf6-f567-48fc-a617-e4df29bb1964`::Granted request >Thread-46::DEBUG::2014-08-22 09:48:23,859::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bc37771f-fa57-4d45-aa3e-932d3b4c7276`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:48:23,859::task::993::Storage.TaskManager.Task::(_decref) Task=`bc37771f-fa57-4d45-aa3e-932d3b4c7276`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:48:23,861::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107683512320', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107679318016', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:23,861::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc37771f-fa57-4d45-aa3e-932d3b4c7276`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107683512320', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107679318016', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:23,861::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc37771f-fa57-4d45-aa3e-932d3b4c7276`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:48:23,862::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:48:23,862::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:48:23,862::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:48:23,862::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:48:23,862::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:48:23,862::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:48:23,862::task::993::Storage.TaskManager.Task::(_decref) Task=`bc37771f-fa57-4d45-aa3e-932d3b4c7276`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:25,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:25,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118303 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:27,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:27,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113966 s, 89.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:28,804::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:28,804::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:28,804::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:28,805::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:28,805::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:28,805::__init__::225::IOProcess::(_processLogs) (1450) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:28,805::__init__::225::IOProcess::(_processLogs) (1450) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:28,805::__init__::225::IOProcess::(_processLogs) (1450) Queuing response >Thread-21::DEBUG::2014-08-22 09:48:28,806::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:28,806::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:28,813::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:48:28,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000146929 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:28,814::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:28,814::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:28,814::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:28,814::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:28,814::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:28,815::__init__::225::IOProcess::(_processLogs) (1451) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:28,815::__init__::225::IOProcess::(_processLogs) (1451) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:28,815::__init__::225::IOProcess::(_processLogs) (1451) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:28,816::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:28,816::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:29,719::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:29,719::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:29,719::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:29,720::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:29,720::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:29,720::__init__::225::IOProcess::(_processLogs) (1452) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:29,720::__init__::225::IOProcess::(_processLogs) (1452) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:29,720::__init__::225::IOProcess::(_processLogs) (1452) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:29,721::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:48:29,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:29,721::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:48:29,727::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000259444 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:29,728::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:29,728::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:29,728::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:29,728::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:29,728::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:29,729::__init__::225::IOProcess::(_processLogs) (1453) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:29,729::__init__::225::IOProcess::(_processLogs) (1453) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:29,729::__init__::225::IOProcess::(_processLogs) (1453) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:29,730::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:29,730::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:29,730::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:48:29,730::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) (1454) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) (1454) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) (1454) Building response >Thread-17::DEBUG::2014-08-22 09:48:29,731::__init__::225::IOProcess::(_processLogs) (1454) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:29,732::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:29,732::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:29,732::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:48:29,732::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:29,732::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) (1455) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) (1455) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) (1455) Building response >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) (1455) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:29,733::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:48:29,734::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:29,734::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:29,734::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) (1456) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) (1456) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) (1456) Building response >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) (1456) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:29,735::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:48:29,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:29,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114926 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:31,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:31,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118428 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:33,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:33,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118751 s, 86.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:33,945::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:48:33,945::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb44693-eb58-4af4-a2a6-8062a99a8096`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:33,945::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:48:33,945::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:33,946::task::1191::Storage.TaskManager.Task::(prepare) Task=`beb44693-eb58-4af4-a2a6-8062a99a8096`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:33,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb44693-eb58-4af4-a2a6-8062a99a8096`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:33,946::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:33,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:33,946::task::993::Storage.TaskManager.Task::(_decref) Task=`beb44693-eb58-4af4-a2a6-8062a99a8096`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:48:33,999::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:48:33,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f54c13d-b2ad-46c7-ba8a-ba379cf27b76`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:48:33,999::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:48:34,000::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`58299094-7f63-430d-9501-a332a868f559`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:48:34,000::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:48:34,000::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:48:34,000::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`58299094-7f63-430d-9501-a332a868f559`::Granted request >Thread-46::DEBUG::2014-08-22 09:48:34,000::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7f54c13d-b2ad-46c7-ba8a-ba379cf27b76`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:48:34,000::task::993::Storage.TaskManager.Task::(_decref) Task=`7f54c13d-b2ad-46c7-ba8a-ba379cf27b76`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:48:34,002::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107679318016', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107662540800', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:34,002::task::1191::Storage.TaskManager.Task::(prepare) Task=`7f54c13d-b2ad-46c7-ba8a-ba379cf27b76`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107679318016', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107662540800', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:34,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f54c13d-b2ad-46c7-ba8a-ba379cf27b76`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:48:34,002::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:48:34,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:48:34,003::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:48:34,003::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:48:34,003::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:48:34,003::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:48:34,003::task::993::Storage.TaskManager.Task::(_decref) Task=`7f54c13d-b2ad-46c7-ba8a-ba379cf27b76`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:35,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:35,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116112 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:37,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d0059f1-efb5-4da8-ad08-5e5f4eedca76`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:37,581::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:48:37,581::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000146929', 'lastCheck': '8.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000259444', 'lastCheck': '7.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:37,581::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d0059f1-efb5-4da8-ad08-5e5f4eedca76`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000146929', 'lastCheck': '8.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000259444', 'lastCheck': '7.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:37,581::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d0059f1-efb5-4da8-ad08-5e5f4eedca76`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:37,581::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:37,581::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:37,581::task::993::Storage.TaskManager.Task::(_decref) Task=`2d0059f1-efb5-4da8-ad08-5e5f4eedca76`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:37,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:37,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196842 s, 52.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:38,817::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:38,817::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:38,818::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:48:38,818::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:38,818::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:38,825::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:48:38,826::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000173913 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:38,825::__init__::225::IOProcess::(_processLogs) (1457) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) (1457) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) (1457) Building response >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) (1457) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:38,827::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:38,828::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:38,828::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:38,828::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:38,829::__init__::225::IOProcess::(_processLogs) (1458) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:38,829::__init__::225::IOProcess::(_processLogs) (1458) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:38,829::__init__::225::IOProcess::(_processLogs) (1458) Building response >Thread-17::DEBUG::2014-08-22 09:48:38,829::__init__::225::IOProcess::(_processLogs) (1458) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:38,829::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:38,830::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:39,735::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:39,735::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:39,735::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:39,736::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:39,736::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:48:39,736::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:39,737::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:39,743::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:48:39,744::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218453 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:39,744::__init__::225::IOProcess::(_processLogs) (1459) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) (1459) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) (1459) Building response >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) (1459) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:39,745::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:39,746::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) (1460) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) (1460) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:39,747::__init__::225::IOProcess::(_processLogs) (1460) Building response >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) (1460) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:39,748::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) (1461) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) (1461) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) (1461) Building response >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) (1461) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:39,749::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:39,750::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:39,750::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:48:39,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:39,750::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:39,750::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:39,750::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) (1462) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) (1462) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) (1462) Building response >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) (1462) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:48:39,751::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:39,752::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:39,752::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:39,752::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) (1463) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) (1463) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) (1463) Building response >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) (1463) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:39,753::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:48:40,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:40,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122579 s, 83.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:42,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:42,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118341 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:44,071::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:48:44,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5909a5b-78e5-4c59-89e7-75dd6087b67a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:44,071::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:48:44,071::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:44,071::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5909a5b-78e5-4c59-89e7-75dd6087b67a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:44,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5909a5b-78e5-4c59-89e7-75dd6087b67a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:44,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:44,072::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:44,072::task::993::Storage.TaskManager.Task::(_decref) Task=`a5909a5b-78e5-4c59-89e7-75dd6087b67a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:44,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-46::DEBUG::2014-08-22 09:48:44,116::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:48:44,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c7dd126-1fb8-4471-b80d-2fb655bbce07`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:48:44,116::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:48:44,117::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`05fb5b1a-4fc3-4894-9946-376d7d8f641f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:48:44,117::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:48:44,117::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:48:44,117::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`05fb5b1a-4fc3-4894-9946-376d7d8f641f`::Granted request >Thread-46::DEBUG::2014-08-22 09:48:44,117::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0c7dd126-1fb8-4471-b80d-2fb655bbce07`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:48:44,117::task::993::Storage.TaskManager.Task::(_decref) Task=`0c7dd126-1fb8-4471-b80d-2fb655bbce07`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:48:44,124::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:44,124::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c7dd126-1fb8-4471-b80d-2fb655bbce07`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:44,125::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c7dd126-1fb8-4471-b80d-2fb655bbce07`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:48:44,125::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:48:44,126::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:48:44,126::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:48:44,127::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:48:44,127::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:48:44,128::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:48:44,129::task::993::Storage.TaskManager.Task::(_decref) Task=`0c7dd126-1fb8-4471-b80d-2fb655bbce07`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:44,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206109 s, 49.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:46,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:46,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116788 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:48,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:48,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011659 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:48,830::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:48,830::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:48,830::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:48,831::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:48:48,831::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:48,831::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:48,838::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:48:48,839::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000167511 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:48,839::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) (1464) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) (1464) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) (1464) Building response >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) (1464) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:48,840::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) (1465) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:48,841::__init__::225::IOProcess::(_processLogs) (1465) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:48,842::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|(1465) Building response >Thread-17::DEBUG::2014-08-22 09:48:48,843::__init__::225::IOProcess::(_processLogs) (1465) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:48,843::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:48,843::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:49,753::__init__::225::IOProcess::(_processLogs) Message size is 161DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:48:49,753::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:49,753::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:49,753::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:49,753::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:49,753::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:49,754::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:49,754::__init__::225::IOProcess::(_processLogs) (1466) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:49,754::__init__::225::IOProcess::(_processLogs) (1466) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:49,754::__init__::225::IOProcess::(_processLogs) (1466) Queuing response >Thread-22::DEBUG::2014-08-22 09:48:49,755::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:49,755::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:49,761::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:48:49,761::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000295044 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) (1467) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:49,762::__init__::225::IOProcess::(_processLogs) (1467) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:49,763::__init__::225::IOProcess::(_processLogs) (1467) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:49,764::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:49,764::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:49,764::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:48:49,764::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:49,764::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) (1468) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) (1468) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) (1468) Building response >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) (1468) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:49,765::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:49,766::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:48:49,766::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:49,766::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:49,766::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) (1469) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) (1469) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) (1469) Building response >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) (1469) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:48:49,767::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:49,768::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:49,768::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:49,768::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:49,768::__init__::225::IOProcess::(_processLogs) (1470) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:49,769::__init__::225::IOProcess::(_processLogs) (1470) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:49,769::__init__::225::IOProcess::(_processLogs) (1470) Building response >Thread-17::DEBUG::2014-08-22 09:48:49,769::__init__::225::IOProcess::(_processLogs) (1470) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:49,769::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:49,769::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:48:50,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:50,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118369 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:52,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:52,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116402 s, 88.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:48:52,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e34bbb2-eb21-419d-8813-3f7f01c83bf7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:52,785::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:48:52,785::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000167511', 'lastCheck': '3.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000295044', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:52,786::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e34bbb2-eb21-419d-8813-3f7f01c83bf7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000167511', 'lastCheck': '3.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000295044', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:48:52,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e34bbb2-eb21-419d-8813-3f7f01c83bf7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:52,786::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:52,786::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:52,786::task::993::Storage.TaskManager.Task::(_decref) Task=`9e34bbb2-eb21-419d-8813-3f7f01c83bf7`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:48:54,345::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:48:54,345::task::595::Storage.TaskManager.Task::(_updateState) Task=`67a7593e-5575-4913-9529-062bdbe4819c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:48:54,345::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:48:54,346::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:54,346::task::1191::Storage.TaskManager.Task::(prepare) Task=`67a7593e-5575-4913-9529-062bdbe4819c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:48:54,346::task::595::Storage.TaskManager.Task::(_updateState) Task=`67a7593e-5575-4913-9529-062bdbe4819c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:48:54,346::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:48:54,346::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:48:54,346::task::993::Storage.TaskManager.Task::(_decref) Task=`67a7593e-5575-4913-9529-062bdbe4819c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:54,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:54,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116353 s, 88.0 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 09:48:54,408::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:48:54,409::task::595::Storage.TaskManager.Task::(_updateState) Task=`3052da9b-0488-4445-b4fd-8e0b98781dac`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:48:54,410::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:48:54,411::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2cabcfae-29ee-4404-b556-638f5299e85f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:48:54,411::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:48:54,412::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:48:54,413::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2cabcfae-29ee-4404-b556-638f5299e85f`::Granted request >Thread-46::DEBUG::2014-08-22 09:48:54,414::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3052da9b-0488-4445-b4fd-8e0b98781dac`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:48:54,414::task::993::Storage.TaskManager.Task::(_decref) Task=`3052da9b-0488-4445-b4fd-8e0b98781dac`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:48:54,423::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:54,424::task::1191::Storage.TaskManager.Task::(prepare) Task=`3052da9b-0488-4445-b4fd-8e0b98781dac`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107661492224', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:48:54,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`3052da9b-0488-4445-b4fd-8e0b98781dac`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:48:54,424::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:48:54,425::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:48:54,425::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:48:54,426::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:48:54,426::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:48:54,427::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:48:54,427::task::993::Storage.TaskManager.Task::(_decref) Task=`3052da9b-0488-4445-b4fd-8e0b98781dac`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:48:56,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:56,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117737 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:48:58,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:48:58,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116641 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:58,843::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:58,843::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:58,843::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:58,843::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:58,844::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:58,844::__init__::225::IOProcess::(_processLogs) (1471) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:58,844::__init__::225::IOProcess::(_processLogs) (1471) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:58,844::__init__::225::IOProcess::(_processLogs) (1471) Queuing response >Thread-21::DEBUG::2014-08-22 09:48:58,845::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:58,845::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:58,851::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:48:58,852::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000183474 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) (1472) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:58,853::__init__::225::IOProcess::(_processLogs) (1472) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:58,854::__init__::225::IOProcess::(_processLogs) (1472) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:58,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:58,855::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:59,769::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:59,769::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:59,769::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:48:59,770::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:48:59,770::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:59,776::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:59,777::__init__::225::IOProcess::(_processLogs) (1473) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:48:59,777::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000280838 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:48:59,778::__init__::225::IOProcess::(_processLogs) (1473) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:59,778::__init__::225::IOProcess::(_processLogs) (1473) Building response >Thread-17::DEBUG::2014-08-22 09:48:59,778::__init__::225::IOProcess::(_processLogs) (1473) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:59,778::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:59,778::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) (1474) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:48:59,779::__init__::225::IOProcess::(_processLogs) (1474) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:48:59,780::__init__::225::IOProcess::(_processLogs) (1474) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:59,780::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:59,780::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:48:59,781::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:48:59,781::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:59,781::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:59,781::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:59,781::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:59,781::__init__::225::IOProcess::(_processLogs) (1475) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) (1475) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) (1475) Building response >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) (1475) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:59,782::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) (1476) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) (1476) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) (1476) Building response >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) (1476) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:48:59,783::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:48:59,784::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:48:59,784::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:48:59,784::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) (1477) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) (1477) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) (1477) Building response >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) (1477) Queuing response >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:48:59,785::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:49:00,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:00,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123215 s, 83.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:02,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:02,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212532 s, 48.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:04,497::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:49:04,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`acdbdc2e-60a7-473e-bf89-3cee904b8307`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:04,498::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:49:04,498::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:04,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`acdbdc2e-60a7-473e-bf89-3cee904b8307`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:04,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`acdbdc2e-60a7-473e-bf89-3cee904b8307`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:04,499::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:04,499::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:04,499::task::993::Storage.TaskManager.Task::(_decref) Task=`acdbdc2e-60a7-473e-bf89-3cee904b8307`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:49:04,522::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:49:04,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`1fcedd0a-7f37-49b8-8046-c347131419c3`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:49:04,523::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:49:04,523::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e7afa40b-e2e1-480f-9f35-db0cdeb4c4a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:49:04,523::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:49:04,523::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:49:04,523::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e7afa40b-e2e1-480f-9f35-db0cdeb4c4a7`::Granted request >Thread-46::DEBUG::2014-08-22 09:49:04,524::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1fcedd0a-7f37-49b8-8046-c347131419c3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:49:04,524::task::993::Storage.TaskManager.Task::(_decref) Task=`1fcedd0a-7f37-49b8-8046-c347131419c3`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:49:04,526::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:04,526::task::1191::Storage.TaskManager.Task::(prepare) Task=`1fcedd0a-7f37-49b8-8046-c347131419c3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:04,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`1fcedd0a-7f37-49b8-8046-c347131419c3`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:49:04,526::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:49:04,526::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:49:04,526::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:49:04,526::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:49:04,526::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:49:04,526::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:49:04,526::task::993::Storage.TaskManager.Task::(_decref) Task=`1fcedd0a-7f37-49b8-8046-c347131419c3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:04,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:04,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188232 s, 54.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:06,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:06,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118377 s, 86.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:08,011::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4ae0559-af9d-4558-924b-37c41d77444c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:08,012::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:49:08,012::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183474', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000280838', 'lastCheck': '8.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:08,012::task::1191::Storage.TaskManager.Task::(prepare) Task=`b4ae0559-af9d-4558-924b-37c41d77444c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183474', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000280838', 'lastCheck': '8.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:08,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4ae0559-af9d-4558-924b-37c41d77444c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:08,012::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:08,012::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:08,012::task::993::Storage.TaskManager.Task::(_decref) Task=`b4ae0559-af9d-4558-924b-37c41d77444c`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:49:08,855::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) (1478) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:08,856::__init__::225::IOProcess::(_processLogs) (1478) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:08,857::__init__::225::IOProcess::(_processLogs) (1478) Queuing response >Thread-21::DEBUG::2014-08-22 09:49:08,858::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:08,858::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:08,864::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:49:08,865::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000156311 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:08,865::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:08,866::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:08,867::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:08,867::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:08,868::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:08,868::__init__::225::IOProcess::(_processLogs) (1479) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:08,868::__init__::225::IOProcess::(_processLogs) (1479) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:08,868::__init__::225::IOProcess::(_processLogs) (1479) Building response >Thread-17::DEBUG::2014-08-22 09:49:08,869::__init__::225::IOProcess::(_processLogs) (1479) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:08,869::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:08,869::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:49:08,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:08,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117521 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:09,785::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:09,785::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:09,786::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:09,786::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:09,786::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:09,786::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:09,786::__init__::225::IOProcess::(_processLogs) (1480) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:09,787::__init__::225::IOProcess::(_processLogs) (1480) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:09,787::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1480) Building response >Thread-22::DEBUG::2014-08-22 09:49:09,788::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:09,788::__init__::225::IOProcess::(_processLogs) (1480) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:09,794::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:09,795::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:49:09,795::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000216401 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:09,796::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:09,797::__init__::225::IOProcess::(_processLogs) (1481) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:09,797::__init__::225::IOProcess::(_processLogs) (1481) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:09,797::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1481) Building response >Thread-17::DEBUG::2014-08-22 09:49:09,798::__init__::225::IOProcess::(_processLogs) (1481) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:09,798::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:09,798::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:09,798::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:09,798::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:49:09,799::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:09,799::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:09,799::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:09,799::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:09,799::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:09,799::__init__::225::IOProcess::(_processLogs) (1482) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) (1482) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) (1482) Building response >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) (1482) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:49:09,800::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) (1483) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) (1483) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) (1483) Building response >Thread-17::DEBUG::2014-08-22 09:49:09,801::__init__::225::IOProcess::(_processLogs) (1483) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:09,802::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:09,802::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:09,802::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:09,802::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:49:09,802::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:09,802::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) (1484) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) (1484) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) (1484) Building response >Thread-17::DEBUG::2014-08-22 09:49:09,803::__init__::225::IOProcess::(_processLogs) (1484) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:09,804::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:09,804::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:49:10,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:10,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117512 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:12,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:13,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115542 s, 88.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:14,582::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:49:14,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`01ed21e9-90d2-4552-b70e-7eaba51454a0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:14,583::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:49:14,583::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:14,583::task::1191::Storage.TaskManager.Task::(prepare) Task=`01ed21e9-90d2-4552-b70e-7eaba51454a0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:14,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`01ed21e9-90d2-4552-b70e-7eaba51454a0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:14,583::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:14,583::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:14,583::task::993::Storage.TaskManager.Task::(_decref) Task=`01ed21e9-90d2-4552-b70e-7eaba51454a0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:49:14,635::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:49:14,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a20e619-7a50-4580-a881-75f326ecd94e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:49:14,635::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:49:14,636::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6c548a40-2179-4783-9c91-04906ea09c01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:49:14,636::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:49:14,636::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:49:14,636::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6c548a40-2179-4783-9c91-04906ea09c01`::Granted request >Thread-46::DEBUG::2014-08-22 09:49:14,636::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4a20e619-7a50-4580-a881-75f326ecd94e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:49:14,636::task::993::Storage.TaskManager.Task::(_decref) Task=`4a20e619-7a50-4580-a881-75f326ecd94e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:49:14,638::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:14,638::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a20e619-7a50-4580-a881-75f326ecd94e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:14,639::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a20e619-7a50-4580-a881-75f326ecd94e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:49:14,639::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:49:14,639::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:49:14,639::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:49:14,639::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:49:14,639::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:49:14,639::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:49:14,639::task::993::Storage.TaskManager.Task::(_decref) Task=`4a20e619-7a50-4580-a881-75f326ecd94e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:15,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:15,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011664 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:17,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:17,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116048 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:18,869::__init__::225::IOProcess::(_processLogs) (1485) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:18,870::__init__::225::IOProcess::(_processLogs) (1485) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:18,870::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1485) Building response >Thread-21::DEBUG::2014-08-22 09:49:18,871::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:18,871::__init__::225::IOProcess::(_processLogs) (1485) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:18,877::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:18,878::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:49:18,878::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000229485 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:18,879::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:18,880::__init__::225::IOProcess::(_processLogs) (1486) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:18,880::__init__::225::IOProcess::(_processLogs) (1486) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:18,880::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1486) Building response >Thread-17::DEBUG::2014-08-22 09:49:18,881::__init__::225::IOProcess::(_processLogs) (1486) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:18,881::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:18,881::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:49:19,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:19,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117766 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:19,803::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) (1487) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:19,804::__init__::225::IOProcess::(_processLogs) (1487) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:19,805::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1487) Building response >Thread-17::DEBUG::2014-08-22 09:49:19,805::__init__::225::IOProcess::(_processLogs) (1487) Queuing response >Thread-22::DEBUG::2014-08-22 09:49:19,805::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:19,806::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:19,812::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:49:19,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000172618 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:19,813::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:19,813::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:19,813::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) (1488) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) (1488) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:19,814::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1488) Building response >Thread-17::DEBUG::2014-08-22 09:49:19,815::__init__::225::IOProcess::(_processLogs) (1488) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:19,815::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) (1489) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:19,816::__init__::225::IOProcess::(_processLogs) (1489) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:19,817::__init__::225::IOProcess::(_processLogs) (1489) Building response >Thread-17::DEBUG::2014-08-22 09:49:19,817::__init__::225::IOProcess::(_processLogs) (1489) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:19,817::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:19,817::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:19,818::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:49:19,818::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:19,818::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:19,818::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) (1490) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) (1490) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) (1490) Building response >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) (1490) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:49:19,819::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:19,820::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:19,820::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) (1491) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) (1491) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) (1491) Building response >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) (1491) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:19,821::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:49:21,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:21,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116953 s, 87.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:23,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b98a013-792c-4e9c-bc3c-8eb52ae6d591`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:23,226::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:49:23,226::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000229485', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000172618', 'lastCheck': '3.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:23,226::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b98a013-792c-4e9c-bc3c-8eb52ae6d591`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000229485', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000172618', 'lastCheck': '3.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:23,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b98a013-792c-4e9c-bc3c-8eb52ae6d591`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:23,227::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:23,227::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:23,227::task::993::Storage.TaskManager.Task::(_decref) Task=`7b98a013-792c-4e9c-bc3c-8eb52ae6d591`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:23,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:23,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01139 s, 89.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:24,697::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:49:24,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`679cdcb7-e772-417e-9b15-69cbdfa41b67`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:24,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:49:24,698::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:24,698::task::1191::Storage.TaskManager.Task::(prepare) Task=`679cdcb7-e772-417e-9b15-69cbdfa41b67`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:24,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`679cdcb7-e772-417e-9b15-69cbdfa41b67`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:24,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:24,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:24,698::task::993::Storage.TaskManager.Task::(_decref) Task=`679cdcb7-e772-417e-9b15-69cbdfa41b67`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:49:24,731::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:49:24,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a9c9a79-1980-42ce-a177-df12fc4be69c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:49:24,731::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:49:24,732::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`414c5cda-32d9-4442-a41e-a2d38de2792b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:49:24,732::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:49:24,732::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:49:24,732::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`414c5cda-32d9-4442-a41e-a2d38de2792b`::Granted request >Thread-46::DEBUG::2014-08-22 09:49:24,732::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8a9c9a79-1980-42ce-a177-df12fc4be69c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:49:24,732::task::993::Storage.TaskManager.Task::(_decref) Task=`8a9c9a79-1980-42ce-a177-df12fc4be69c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:49:24,734::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:24,734::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a9c9a79-1980-42ce-a177-df12fc4be69c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:24,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a9c9a79-1980-42ce-a177-df12fc4be69c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:49:24,734::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:49:24,735::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:49:24,735::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:49:24,735::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:49:24,735::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:49:24,735::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:49:24,735::task::993::Storage.TaskManager.Task::(_decref) Task=`8a9c9a79-1980-42ce-a177-df12fc4be69c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:25,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:25,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117426 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:27,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:27,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115339 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:28,882::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:28,882::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:28,883::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:49:28,883::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:28,883::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:28,889::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:28,890::__init__::225::IOProcess::(_processLogs) (1492) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:49:28,891::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000206507 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:28,891::__init__::225::IOProcess::(_processLogs) (1492) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:28,892::__init__::225::IOProcess::(_processLogs) (1492) Building response >Thread-17::DEBUG::2014-08-22 09:49:28,892::__init__::225::IOProcess::(_processLogs) (1492) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:28,892::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:28,892::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) (1493) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:28,893::__init__::225::IOProcess::(_processLogs) (1493) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:28,894::__init__::225::IOProcess::(_processLogs) (1493) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:28,895::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:28,895::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:49:29,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:29,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117566 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:29,821::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:29,821::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:29,821::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:29,821::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:29,821::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:29,822::__init__::225::IOProcess::(_processLogs) (1494) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:29,822::__init__::225::IOProcess::(_processLogs) (1494) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:29,832::__init__::225::IOProcess::(_processLogs) (1494) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:29,833::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:49:29,833::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:49:29,910::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00064253 s, 887 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) (1495) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:29,911::__init__::225::IOProcess::(_processLogs) (1495) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:29,912::__init__::225::IOProcess::(_processLogs) (1495) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:29,913::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:29,913::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:29,913::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:49:29,913::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:29,913::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) (1496) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) (1496) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) (1496) Building response >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) (1496) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:29,914::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:49:29,915::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:29,915::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:29,915::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:29,915::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:29,915::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) (1497) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) (1497) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) (1497) Building response >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) (1497) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:49:29,916::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:29,917::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:29,917::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:29,917::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) (1498) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) (1498) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) (1498) Building response >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) (1498) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:29,918::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:49:31,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:31,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119753 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:33,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:33,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115741 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:34,788::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:49:34,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebd9d615-8073-4e8e-9701-1b3cd6f5ce6f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:34,788::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:49:34,789::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:34,789::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebd9d615-8073-4e8e-9701-1b3cd6f5ce6f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:34,789::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebd9d615-8073-4e8e-9701-1b3cd6f5ce6f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:34,789::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:34,789::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:34,789::task::993::Storage.TaskManager.Task::(_decref) Task=`ebd9d615-8073-4e8e-9701-1b3cd6f5ce6f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:49:34,826::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:49:34,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`db440e76-9a60-4cd2-bc2f-508ef848f59c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:49:34,827::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:49:34,827::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b06c0b7c-5acb-4e66-904e-28fe2c89ddb3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:49:34,827::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:49:34,827::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:49:34,827::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b06c0b7c-5acb-4e66-904e-28fe2c89ddb3`::Granted request >Thread-46::DEBUG::2014-08-22 09:49:34,828::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`db440e76-9a60-4cd2-bc2f-508ef848f59c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:49:34,828::task::993::Storage.TaskManager.Task::(_decref) Task=`db440e76-9a60-4cd2-bc2f-508ef848f59c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:49:34,830::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107653103616', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:34,830::task::1191::Storage.TaskManager.Task::(prepare) Task=`db440e76-9a60-4cd2-bc2f-508ef848f59c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107660443648', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107653103616', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:34,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`db440e76-9a60-4cd2-bc2f-508ef848f59c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:49:34,830::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:49:34,830::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:49:34,830::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:49:34,830::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:49:34,830::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:49:34,830::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:49:34,830::task::993::Storage.TaskManager.Task::(_decref) Task=`db440e76-9a60-4cd2-bc2f-508ef848f59c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:35,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:35,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117205 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:37,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:37,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115736 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:38,494::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9c48a86-a6f8-4fcb-9e17-f7fc4303f35b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:38,494::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:49:38,495::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206507', 'lastCheck': '9.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00064253', 'lastCheck': '8.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:38,495::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9c48a86-a6f8-4fcb-9e17-f7fc4303f35b`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206507', 'lastCheck': '9.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00064253', 'lastCheck': '8.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:38,495::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9c48a86-a6f8-4fcb-9e17-f7fc4303f35b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:38,495::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:38,495::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:38,495::task::993::Storage.TaskManager.Task::(_decref) Task=`b9c48a86-a6f8-4fcb-9e17-f7fc4303f35b`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:49:38,895::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:38,896::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:38,896::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:38,896::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:49:38,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:38,897::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:38,903::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:38,904::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:49:38,904::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000185344 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:38,905::__init__::225::IOProcess::(_processLogs) (1499) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:38,905::__init__::225::IOProcess::(_processLogs) (1499) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:38,905::__init__::225::IOProcess::(_processLogs) (1499) Building response >Thread-17::DEBUG::2014-08-22 09:49:38,905::__init__::225::IOProcess::(_processLogs) (1499) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:38,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:38,907::__init__::225::IOProcess::(_processLogs) (1500) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:38,907::__init__::225::IOProcess::(_processLogs) (1500) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:38,907::__init__::225::IOProcess::(_processLogs) (1500) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:38,908::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:38,908::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:49:39,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:39,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230836 s, 44.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) (1501) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:39,918::__init__::225::IOProcess::(_processLogs) (1501) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:39,919::__init__::225::IOProcess::(_processLogs) (1501) Queuing response >Thread-22::DEBUG::2014-08-22 09:49:39,920::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:39,920::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:39,926::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:49:39,927::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00028353 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:39,927::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:39,928::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:39,928::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:39,928::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:39,928::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:39,928::__init__::225::IOProcess::(_processLogs) (1502) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:39,928::__init__::225::IOProcess::(_processLogs) (1502) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:39,929::__init__::225::IOProcess::(_processLogs) (1502) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:39,929::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:39,929::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:39,930::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:49:39,930::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:39,930::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:39,930::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:39,930::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:39,930::__init__::225::IOProcess::(_processLogs) (1503) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) (1503) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) (1503) Building response >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) (1503) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:49:39,931::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) (1504) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) (1504) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) (1504) Building response >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) (1504) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:39,932::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:39,933::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:49:39,933::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:39,933::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:39,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) (1505) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) (1505) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) (1505) Building response >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) (1505) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:39,934::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:49:41,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:41,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117531 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:43,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:43,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117136 s, 87.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:44,906::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:49:44,906::task::595::Storage.TaskManager.Task::(_updateState) Task=`864ce96e-2a08-419b-8071-e9700d086165`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:44,906::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:49:44,907::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:44,907::task::1191::Storage.TaskManager.Task::(prepare) Task=`864ce96e-2a08-419b-8071-e9700d086165`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:44,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`864ce96e-2a08-419b-8071-e9700d086165`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:44,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:44,907::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:44,907::task::993::Storage.TaskManager.Task::(_decref) Task=`864ce96e-2a08-419b-8071-e9700d086165`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:49:44,965::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:49:44,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`088256e7-edf9-474b-9fdd-5719b7fa229b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:49:44,966::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:49:44,966::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b0956a33-e0d1-4577-b347-5be064623366`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:49:44,966::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:49:44,966::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:49:44,966::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b0956a33-e0d1-4577-b347-5be064623366`::Granted request >Thread-46::DEBUG::2014-08-22 09:49:44,967::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`088256e7-edf9-474b-9fdd-5719b7fa229b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:49:44,967::task::993::Storage.TaskManager.Task::(_decref) Task=`088256e7-edf9-474b-9fdd-5719b7fa229b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:49:44,969::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:44,969::task::1191::Storage.TaskManager.Task::(prepare) Task=`088256e7-edf9-474b-9fdd-5719b7fa229b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:44,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`088256e7-edf9-474b-9fdd-5719b7fa229b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:49:44,969::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:49:44,969::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:49:44,969::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:49:44,969::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:49:44,969::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:49:44,969::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:49:44,969::task::993::Storage.TaskManager.Task::(_decref) Task=`088256e7-edf9-474b-9fdd-5719b7fa229b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:45,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:45,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011541 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:47,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:48,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116397 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:48,908::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:48,909::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:48,909::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:49:48,910::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:48,910::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:48,916::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:48,917::__init__::225::IOProcess::(_processLogs) (1506) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:49:48,917::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000206111 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:48,918::__init__::225::IOProcess::(_processLogs) (1506) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:48,918::__init__::225::IOProcess::(_processLogs) (1506) Building response >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) (1506) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:48,919::__init__::225::IOProcess::(_processLogs) (1507) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:48,920::__init__::225::IOProcess::(_processLogs) (1507) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:48,920::__init__::225::IOProcess::(_processLogs) (1507) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:48,921::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:48,921::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:49,934::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:49,934::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:49,934::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:49,935::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:49,935::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:49,935::__init__::225::IOProcess::(_processLogs) (1508) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:49,935::__init__::225::IOProcess::(_processLogs) (1508) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:49,935::__init__::225::IOProcess::(_processLogs) (1508) Queuing response >Thread-22::DEBUG::2014-08-22 09:49:49,936::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:49,936::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:49,942::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:49:49,943::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200945 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) (1509) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:49,944::__init__::225::IOProcess::(_processLogs) (1509) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:49,945::__init__::225::IOProcess::(_processLogs) (1509) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:49,946::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:49,946::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:49,946::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:49:49,946::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:49,946::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) (1510) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) (1510) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) (1510) Building response >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) (1510) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:49,947::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:49,948::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:49:49,948::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:49,948::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:49,948::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:49,948::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:49,948::__init__::225::IOProcess::(_processLogs) (1511) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) (1511) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) (1511) Building response >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) (1511) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:49:49,949::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:49,950::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:49,950::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:49,950::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:49,950::__init__::225::IOProcess::(_processLogs) (1512) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:49,950::__init__::225::IOProcess::(_processLogs) (1512) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:49,951::__init__::225::IOProcess::(_processLogs) (1512) Building response >Thread-17::DEBUG::2014-08-22 09:49:49,951::__init__::225::IOProcess::(_processLogs) (1512) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:49,951::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:49,951::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:49:50,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:50,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120574 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:52,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:52,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121469 s, 84.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:53,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`a087a11b-9421-40ab-a3b8-c744962f2b02`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:53,698::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:49:53,698::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206111', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000200945', 'lastCheck': '3.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:53,698::task::1191::Storage.TaskManager.Task::(prepare) Task=`a087a11b-9421-40ab-a3b8-c744962f2b02`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206111', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000200945', 'lastCheck': '3.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:49:53,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`a087a11b-9421-40ab-a3b8-c744962f2b02`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:53,699::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:53,699::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:53,699::task::993::Storage.TaskManager.Task::(_decref) Task=`a087a11b-9421-40ab-a3b8-c744962f2b02`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:54,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:54,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119595 s, 85.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:49:55,079::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-47::DEBUG::2014-08-22 09:49:55,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`6409bd40-2a86-4bc5-a4f3-7afd2fbc340d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:49:55,080::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:49:55,080::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:55,080::task::1191::Storage.TaskManager.Task::(prepare) Task=`6409bd40-2a86-4bc5-a4f3-7afd2fbc340d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:49:55,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`6409bd40-2a86-4bc5-a4f3-7afd2fbc340d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:49:55,080::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:49:55,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:49:55,080::task::993::Storage.TaskManager.Task::(_decref) Task=`6409bd40-2a86-4bc5-a4f3-7afd2fbc340d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:49:55,117::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [781c67a0] >Thread-46::DEBUG::2014-08-22 09:49:55,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7856292-ac58-44d4-914b-a632b50b9860`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:49:55,117::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:49:55,118::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1b6f304b-5fd0-40f7-a984-7b63e64747f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:49:55,118::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:49:55,118::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:49:55,118::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1b6f304b-5fd0-40f7-a984-7b63e64747f6`::Granted request >Thread-46::DEBUG::2014-08-22 09:49:55,118::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c7856292-ac58-44d4-914b-a632b50b9860`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:49:55,118::task::993::Storage.TaskManager.Task::(_decref) Task=`c7856292-ac58-44d4-914b-a632b50b9860`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:49:55,120::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:55,120::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7856292-ac58-44d4-914b-a632b50b9860`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:49:55,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7856292-ac58-44d4-914b-a632b50b9860`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:49:55,121::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:49:55,121::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:49:55,121::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:49:55,121::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:49:55,121::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:49:55,121::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:49:55,121::task::993::Storage.TaskManager.Task::(_decref) Task=`c7856292-ac58-44d4-914b-a632b50b9860`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:49:56,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:56,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116874 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:49:58,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:49:58,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115171 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:58,921::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:58,922::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:58,922::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:58,923::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:49:58,923::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:58,924::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:58,930::__init__::225::IOProcess::(_processLogs) (1513) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 09:49:58,930::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189914 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) (1513) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) (1513) Building response >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) (1513) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:58,932::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:58,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:58,933::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:58,933::__init__::225::IOProcess::(_processLogs) (1514) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:58,933::__init__::225::IOProcess::(_processLogs) (1514) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:58,933::__init__::225::IOProcess::(_processLogs) (1514) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:58,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:58,934::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) (1515) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:59,951::__init__::225::IOProcess::(_processLogs) (1515) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:59,952::__init__::225::IOProcess::(_processLogs) (1515) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:59,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:49:59,952::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:49:59,953::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:49:59,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000189572 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) (1516) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:49:59,960::__init__::225::IOProcess::(_processLogs) (1516) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:49:59,961::__init__::225::IOProcess::(_processLogs) (1516) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:59,962::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:59,962::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:49:59,962::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:49:59,962::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:59,962::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) (1517) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) (1517) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) (1517) Building response >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) (1517) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:49:59,963::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:59,964::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:59,964::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:59,964::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:59,964::__init__::225::IOProcess::(_processLogs) (1518) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:59,964::__init__::225::IOProcess::(_processLogs) (1518) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:59,964::__init__::225::IOProcess::(_processLogs) (1518) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) (1519) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:49:59,965::__init__::225::IOProcess::(_processLogs) (1519) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:49:59,966::__init__::225::IOProcess::(_processLogs) (1519) Queuing response >Thread-17::DEBUG::2014-08-22 09:49:59,966::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:49:59,967::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:50:00,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:00,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116022 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:02,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:02,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211112 s, 48.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:04,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:04,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115372 s, 88.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:05,163::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:50:05,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c66a250-a2d4-4362-9fff-26229ff2bc40`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:05,163::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:50:05,163::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:05,164::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c66a250-a2d4-4362-9fff-26229ff2bc40`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:05,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c66a250-a2d4-4362-9fff-26229ff2bc40`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:05,164::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:05,164::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:05,164::task::993::Storage.TaskManager.Task::(_decref) Task=`0c66a250-a2d4-4362-9fff-26229ff2bc40`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:50:05,188::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:50:05,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce76b84b-c700-41de-b284-fa2f875fd0b5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:50:05,188::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:50:05,189::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`11899936-f0b7-475f-9a87-088adee28ef1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:50:05,189::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:50:05,189::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:50:05,189::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`11899936-f0b7-475f-9a87-088adee28ef1`::Granted request >Thread-46::DEBUG::2014-08-22 09:50:05,189::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ce76b84b-c700-41de-b284-fa2f875fd0b5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:50:05,189::task::993::Storage.TaskManager.Task::(_decref) Task=`ce76b84b-c700-41de-b284-fa2f875fd0b5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:50:05,191::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:05,191::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce76b84b-c700-41de-b284-fa2f875fd0b5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107482185728', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:05,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce76b84b-c700-41de-b284-fa2f875fd0b5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:50:05,191::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:50:05,192::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:50:05,192::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:50:05,192::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:50:05,192::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:50:05,192::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:50:05,192::task::993::Storage.TaskManager.Task::(_decref) Task=`ce76b84b-c700-41de-b284-fa2f875fd0b5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:06,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:06,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117182 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:08,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:08,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167325 s, 61.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:08,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`df408a0d-f667-40f1-878e-7ed56e1b1e2d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:08,873::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:50:08,873::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189914', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000189572', 'lastCheck': '8.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:08,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`df408a0d-f667-40f1-878e-7ed56e1b1e2d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189914', 'lastCheck': '9.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000189572', 'lastCheck': '8.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:08,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`df408a0d-f667-40f1-878e-7ed56e1b1e2d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:08,874::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:08,874::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:08,874::task::993::Storage.TaskManager.Task::(_decref) Task=`df408a0d-f667-40f1-878e-7ed56e1b1e2d`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) (1520) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:08,935::__init__::225::IOProcess::(_processLogs) (1520) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:08,936::__init__::225::IOProcess::(_processLogs) (1520) Queuing response >Thread-21::DEBUG::2014-08-22 09:50:08,937::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:08,937::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:08,942::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:50:08,943::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000156403 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:08,943::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:08,943::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:08,944::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:08,944::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:08,944::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:08,944::__init__::225::IOProcess::(_processLogs) (1521) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:08,944::__init__::225::IOProcess::(_processLogs) (1521) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:08,945::__init__::225::IOProcess::(_processLogs) (1521) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:08,945::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:08,946::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) (1522) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:09,967::__init__::225::IOProcess::(_processLogs) (1522) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:09,968::__init__::225::IOProcess::(_processLogs) (1522) Queuing response >Thread-22::DEBUG::2014-08-22 09:50:09,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:09,969::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:09,975::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:50:09,976::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000188109 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:09,976::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:09,977::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:09,977::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:09,977::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:09,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:09,977::__init__::225::IOProcess::(_processLogs) (1523) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:09,977::__init__::225::IOProcess::(_processLogs) (1523) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:09,978::__init__::225::IOProcess::(_processLogs) (1523) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:09,978::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:09,978::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:09,978::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:50:09,979::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:09,979::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:09,979::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:09,979::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:09,979::__init__::225::IOProcess::(_processLogs) (1524) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) (1524) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) (1524) Building response >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) (1524) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:09,980::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) (1525) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) (1525) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) (1525) Building response >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) (1525) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:09,981::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:09,982::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:50:09,982::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:09,982::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:09,982::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) (1526) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) (1526) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) (1526) Building response >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) (1526) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:09,983::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:50:10,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:10,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120172 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:12,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:12,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115311 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:14,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:14,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118101 s, 86.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:15,241::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:50:15,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b097ac8-d71a-4d64-a3f1-1717f9db4a2a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:15,242::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:50:15,242::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:15,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b097ac8-d71a-4d64-a3f1-1717f9db4a2a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:15,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b097ac8-d71a-4d64-a3f1-1717f9db4a2a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:15,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:15,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:15,242::task::993::Storage.TaskManager.Task::(_decref) Task=`4b097ac8-d71a-4d64-a3f1-1717f9db4a2a`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:50:15,327::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:50:15,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6b81796-50c0-402d-8611-186fe4f0f512`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:50:15,328::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:50:15,328::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6c61bfd7-4c22-4a27-ae6f-7457a7047e3f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:50:15,328::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:50:15,328::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:50:15,328::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6c61bfd7-4c22-4a27-ae6f-7457a7047e3f`::Granted request >Thread-46::DEBUG::2014-08-22 09:50:15,329::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a6b81796-50c0-402d-8611-186fe4f0f512`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:50:15,329::task::993::Storage.TaskManager.Task::(_decref) Task=`a6b81796-50c0-402d-8611-186fe4f0f512`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:50:15,331::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:15,331::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6b81796-50c0-402d-8611-186fe4f0f512`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:15,331::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6b81796-50c0-402d-8611-186fe4f0f512`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:50:15,331::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:50:15,331::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:50:15,331::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:50:15,331::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:50:15,331::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:50:15,331::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:50:15,331::task::993::Storage.TaskManager.Task::(_decref) Task=`a6b81796-50c0-402d-8611-186fe4f0f512`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:16,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:16,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116725 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:18,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:18,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117371 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:18,946::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:18,946::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:18,946::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:18,946::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:18,946::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:18,946::__init__::225::IOProcess::(_processLogs) (1527) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:18,947::__init__::225::IOProcess::(_processLogs) (1527) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:18,947::__init__::225::IOProcess::(_processLogs) (1527) Queuing response >Thread-21::DEBUG::2014-08-22 09:50:18,948::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:18,948::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:18,954::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:50:18,954::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178955 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) (1528) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:18,955::__init__::225::IOProcess::(_processLogs) (1528) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:18,956::__init__::225::IOProcess::(_processLogs) (1528) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:18,957::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:18,957::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:19,983::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:19,983::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:19,983::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:19,983::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:19,983::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:19,984::__init__::225::IOProcess::(_processLogs) (1529) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:19,984::__init__::225::IOProcess::(_processLogs) (1529) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:19,984::__init__::225::IOProcess::(_processLogs) (1529) Queuing response >Thread-22::DEBUG::2014-08-22 09:50:19,985::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:19,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:19,991::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:50:19,992::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000147147 s, 3.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:19,992::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:19,992::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:19,992::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:19,993::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:19,993::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:19,993::__init__::225::IOProcess::(_processLogs) (1530) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:19,993::__init__::225::IOProcess::(_processLogs) (1530) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:19,993::__init__::225::IOProcess::(_processLogs) (1530) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:19,994::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:19,994::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:19,994::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:50:19,994::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:19,995::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:19,995::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:19,995::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:19,995::__init__::225::IOProcess::(_processLogs) (1531) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:19,995::__init__::225::IOProcess::(_processLogs) (1531) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:19,995::__init__::225::IOProcess::(_processLogs) (1531) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:19,996::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:19,996::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:19,996::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:50:19,996::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:19,996::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) (1532) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) (1532) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) (1532) Building response >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) (1532) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:19,997::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:50:19,998::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:19,998::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:19,998::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) (1533) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) (1533) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) (1533) Building response >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) (1533) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:19,999::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:50:20,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:20,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118949 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:22,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:23,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116063 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:24,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1177955-5927-4d1b-b40f-7de06cf4d208`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:24,133::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:50:24,134::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178955', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000147147', 'lastCheck': '4.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:24,134::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1177955-5927-4d1b-b40f-7de06cf4d208`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178955', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000147147', 'lastCheck': '4.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:24,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1177955-5927-4d1b-b40f-7de06cf4d208`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:24,134::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:24,134::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:24,134::task::993::Storage.TaskManager.Task::(_decref) Task=`d1177955-5927-4d1b-b40f-7de06cf4d208`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:25,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:25,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0126334 s, 81.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:25,412::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:50:25,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`e63803a7-dbbb-467e-903d-25fbd483747c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:25,412::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:50:25,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:25,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`e63803a7-dbbb-467e-903d-25fbd483747c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:25,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`e63803a7-dbbb-467e-903d-25fbd483747c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:25,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:25,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:25,413::task::993::Storage.TaskManager.Task::(_decref) Task=`e63803a7-dbbb-467e-903d-25fbd483747c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:50:25,442::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:50:25,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d68967c-d548-40d3-9bec-4d89574000d5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:50:25,443::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:50:25,443::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`610ed4eb-9be4-47d7-ac2d-90742259b4a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:50:25,443::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:50:25,443::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:50:25,443::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`610ed4eb-9be4-47d7-ac2d-90742259b4a0`::Granted request >Thread-46::DEBUG::2014-08-22 09:50:25,443::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3d68967c-d548-40d3-9bec-4d89574000d5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:50:25,444::task::993::Storage.TaskManager.Task::(_decref) Task=`3d68967c-d548-40d3-9bec-4d89574000d5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:50:25,445::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:25,446::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d68967c-d548-40d3-9bec-4d89574000d5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:25,446::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d68967c-d548-40d3-9bec-4d89574000d5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:50:25,446::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:50:25,446::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:50:25,446::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:50:25,446::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:50:25,446::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:50:25,446::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:50:25,446::task::993::Storage.TaskManager.Task::(_decref) Task=`3d68967c-d548-40d3-9bec-4d89574000d5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:27,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:27,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116746 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:28,957::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:28,958::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:28,958::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:50:28,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:28,959::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:28,965::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:50:28,966::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000201023 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:28,966::__init__::225::IOProcess::(_processLogs) (1534) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) (1534) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) (1534) Building response >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) (1534) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:28,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:28,969::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:28,969::__init__::225::IOProcess::(_processLogs) (1535) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:28,969::__init__::225::IOProcess::(_processLogs) (1535) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:28,969::__init__::225::IOProcess::(_processLogs) (1535) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:28,970::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:28,971::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:50:29,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:29,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122784 s, 83.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:29,999::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:29,999::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:29,999::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:29,999::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:29,999::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:30,000::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:30,000::__init__::225::IOProcess::(_processLogs) (1536) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:30,000::__init__::225::IOProcess::(_processLogs) (1536) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:30,000::__init__::225::IOProcess::(_processLogs) (1536) Queuing response >Thread-22::DEBUG::2014-08-22 09:50:30,001::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:30,001::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:30,007::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:50:30,008::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000263743 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) (1537) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:30,009::__init__::225::IOProcess::(_processLogs) (1537) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:30,010::__init__::225::IOProcess::(_processLogs) (1537) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:30,010::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:30,011::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:30,011::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:50:30,011::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:30,011::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) (1538) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) (1538) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) (1538) Building response >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) (1538) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:50:30,012::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:30,013::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:30,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:30,013::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:30,013::__init__::225::IOProcess::(_processLogs) (1539) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:30,013::__init__::225::IOProcess::(_processLogs) (1539) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:30,013::__init__::225::IOProcess::(_processLogs) (1539) Building response >Thread-17::DEBUG::2014-08-22 09:50:30,014::__init__::225::IOProcess::(_processLogs) (1539) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:30,014::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:30,014::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:30,014::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:50:30,014::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:30,014::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) (1540) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) (1540) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) (1540) Building response >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) (1540) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:30,015::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:30,016::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:50:31,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:31,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117991 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:33,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:33,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125658 s, 81.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:35,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:35,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117306 s, 87.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:35,511::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:50:35,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`a615b2e7-24db-498d-93b7-0f0cfe3ffab1`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:35,512::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:50:35,512::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:35,512::task::1191::Storage.TaskManager.Task::(prepare) Task=`a615b2e7-24db-498d-93b7-0f0cfe3ffab1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:35,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`a615b2e7-24db-498d-93b7-0f0cfe3ffab1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:35,512::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:35,512::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:35,512::task::993::Storage.TaskManager.Task::(_decref) Task=`a615b2e7-24db-498d-93b7-0f0cfe3ffab1`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:50:35,536::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:50:35,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fda7ebe-aed8-42dc-91b4-a7f2d6330303`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:50:35,537::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:50:35,537::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6d57748b-7449-4e01-bb71-09bc80981ac8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:50:35,537::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:50:35,537::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:50:35,537::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6d57748b-7449-4e01-bb71-09bc80981ac8`::Granted request >Thread-46::DEBUG::2014-08-22 09:50:35,537::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3fda7ebe-aed8-42dc-91b4-a7f2d6330303`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:50:35,538::task::993::Storage.TaskManager.Task::(_decref) Task=`3fda7ebe-aed8-42dc-91b4-a7f2d6330303`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:50:35,539::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:35,540::task::1191::Storage.TaskManager.Task::(prepare) Task=`3fda7ebe-aed8-42dc-91b4-a7f2d6330303`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:35,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fda7ebe-aed8-42dc-91b4-a7f2d6330303`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:50:35,540::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:50:35,540::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:50:35,540::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:50:35,540::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:50:35,540::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:50:35,540::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:50:35,540::task::993::Storage.TaskManager.Task::(_decref) Task=`3fda7ebe-aed8-42dc-91b4-a7f2d6330303`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:37,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:37,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118198 s, 86.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:38,971::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:38,971::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:38,972::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:50:38,973::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:38,973::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:38,979::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:50:38,980::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187353 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:38,980::__init__::225::IOProcess::(_processLogs) (1541) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) (1541) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) (1541) Building response >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) (1541) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:38,981::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:38,982::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:38,982::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:38,982::__init__::225::IOProcess::(_processLogs) (1542) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:38,982::__init__::225::IOProcess::(_processLogs) (1542) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:38,982::__init__::225::IOProcess::(_processLogs) (1542) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:38,983::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:38,983::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:50:39,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-47::DEBUG::2014-08-22 09:50:39,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`37208fb6-584a-402c-a532-1d90edb4717d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:39,392::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:50:39,392::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187353', 'lastCheck': '0.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000263743', 'lastCheck': '9.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:39,393::task::1191::Storage.TaskManager.Task::(prepare) Task=`37208fb6-584a-402c-a532-1d90edb4717d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187353', 'lastCheck': '0.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000263743', 'lastCheck': '9.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:39,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`37208fb6-584a-402c-a532-1d90edb4717d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:39,393::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:39,393::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:39,393::task::993::Storage.TaskManager.Task::(_decref) Task=`37208fb6-584a-402c-a532-1d90edb4717d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:39,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117963 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:40,015::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) (1543) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) (1543) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:40,016::__init__::225::IOProcess::(_processLogs) (1543) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:40,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:50:40,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:40,018::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:50:40,024::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200597 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:40,024::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:40,024::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:40,025::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:40,025::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:40,025::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:40,025::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:40,025::__init__::225::IOProcess::(_processLogs) (1544) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:40,025::__init__::225::IOProcess::(_processLogs) (1544) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:40,026::__init__::225::IOProcess::(_processLogs) (1544) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:40,026::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:40,026::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:40,027::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:50:40,027::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:40,027::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:40,027::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:40,027::__init__::225::IOProcess::(_processLogs) (1545) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) (1545) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) (1545) Building response >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) (1545) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:50:40,028::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) (1546) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) (1546) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) (1546) Building response >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) (1546) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:40,029::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:40,030::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:50:40,030::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:40,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:40,030::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) (1547) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) (1547) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) (1547) Building response >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) (1547) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:40,031::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:50:41,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:41,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119106 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:43,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:43,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115056 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:45,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:45,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118478 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:45,604::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:50:45,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f2a3f42-a446-418c-9e3a-a0aa8e7988ae`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:45,604::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:50:45,605::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:45,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f2a3f42-a446-418c-9e3a-a0aa8e7988ae`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:45,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f2a3f42-a446-418c-9e3a-a0aa8e7988ae`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:45,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:45,605::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:45,605::task::993::Storage.TaskManager.Task::(_decref) Task=`5f2a3f42-a446-418c-9e3a-a0aa8e7988ae`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:50:45,692::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:50:45,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`21356900-b960-45b2-a138-53f2a056d96b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:50:45,692::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:50:45,692::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bead1a3c-c6d1-4c8d-9f28-dd812995d93d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:50:45,692::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:50:45,693::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:50:45,693::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bead1a3c-c6d1-4c8d-9f28-dd812995d93d`::Granted request >Thread-46::DEBUG::2014-08-22 09:50:45,693::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`21356900-b960-45b2-a138-53f2a056d96b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:50:45,693::task::993::Storage.TaskManager.Task::(_decref) Task=`21356900-b960-45b2-a138-53f2a056d96b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:50:45,695::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:45,695::task::1191::Storage.TaskManager.Task::(prepare) Task=`21356900-b960-45b2-a138-53f2a056d96b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107474845696', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:45,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`21356900-b960-45b2-a138-53f2a056d96b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:50:45,695::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:50:45,695::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:50:45,696::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:50:45,696::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:50:45,696::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:50:45,696::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:50:45,696::task::993::Storage.TaskManager.Task::(_decref) Task=`21356900-b960-45b2-a138-53f2a056d96b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:47,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:47,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116095 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:48,984::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:48,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:48,984::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:48,984::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:48,984::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:48,985::__init__::225::IOProcess::(_processLogs) (1548) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:48,985::__init__::225::IOProcess::(_processLogs) (1548) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:48,985::__init__::225::IOProcess::(_processLogs) (1548) Queuing response >Thread-21::DEBUG::2014-08-22 09:50:48,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:48,989::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:48,995::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:50:48,998::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00287807 s, 138 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:48,998::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:48,999::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:48,999::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:48,999::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:48,999::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:48,999::__init__::225::IOProcess::(_processLogs) (1549) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:48,999::__init__::225::IOProcess::(_processLogs) (1549) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:49,001::__init__::225::IOProcess::(_processLogs) (1549) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:49,002::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:49,002::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:50:49,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:49,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01191 s, 86.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:50,031::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:50,031::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:50,031::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:50,031::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:50,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:50,032::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:50,032::__init__::225::IOProcess::(_processLogs) (1550) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:50,032::__init__::225::IOProcess::(_processLogs) (1550) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:50,032::__init__::225::IOProcess::(_processLogs) (1550) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:50,033::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:50:50,033::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:50,033::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:50:50,039::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021845 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:50,040::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:50:50,040::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:50,040::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:50,040::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:50,040::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:50,041::__init__::225::IOProcess::(_processLogs) (1551) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:50,041::__init__::225::IOProcess::(_processLogs) (1551) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:50,041::__init__::225::IOProcess::(_processLogs) (1551) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:50,042::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:50,042::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:50,042::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:50:50,042::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:50,043::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:50,043::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:50,043::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:50,043::__init__::225::IOProcess::(_processLogs) (1552) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:50,043::__init__::225::IOProcess::(_processLogs) (1552) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:50,043::__init__::225::IOProcess::(_processLogs) (1552) Building response >Thread-17::DEBUG::2014-08-22 09:50:50,044::__init__::225::IOProcess::(_processLogs) (1552) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:50,044::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:50,044::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:50,044::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:50:50,044::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:50,044::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) (1553) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) (1553) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) (1553) Building response >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) (1553) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:50:50,045::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:50:50,046::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:50,046::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) (1554) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) (1554) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) (1554) Building response >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) (1554) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:50,047::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:50:51,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:51,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119386 s, 85.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:53,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:53,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118652 s, 86.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:50:54,629::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0cb1914-17a8-47c6-a4d8-bb2ac9dbdc8c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:54,629::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:50:54,629::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00287807', 'lastCheck': '5.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021845', 'lastCheck': '4.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:54,629::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0cb1914-17a8-47c6-a4d8-bb2ac9dbdc8c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00287807', 'lastCheck': '5.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021845', 'lastCheck': '4.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:50:54,630::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0cb1914-17a8-47c6-a4d8-bb2ac9dbdc8c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:54,630::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:54,630::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:54,630::task::993::Storage.TaskManager.Task::(_decref) Task=`e0cb1914-17a8-47c6-a4d8-bb2ac9dbdc8c`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:50:55,763::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-47::DEBUG::2014-08-22 09:50:55,763::task::595::Storage.TaskManager.Task::(_updateState) Task=`29abafa6-ea8d-4608-8d82-235b82a57256`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:50:55,763::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:50:55,763::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:55,763::task::1191::Storage.TaskManager.Task::(prepare) Task=`29abafa6-ea8d-4608-8d82-235b82a57256`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:50:55,764::task::595::Storage.TaskManager.Task::(_updateState) Task=`29abafa6-ea8d-4608-8d82-235b82a57256`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:50:55,764::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:50:55,764::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:50:55,764::task::993::Storage.TaskManager.Task::(_decref) Task=`29abafa6-ea8d-4608-8d82-235b82a57256`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:50:55,786::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [b495545] >Thread-46::DEBUG::2014-08-22 09:50:55,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`64cfb026-93d8-4d68-9aff-94391b27c183`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:50:55,787::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:50:55,787::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`360acd05-693f-4bc7-b6e7-e3687e00f34b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:50:55,787::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:50:55,787::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:50:55,787::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`360acd05-693f-4bc7-b6e7-e3687e00f34b`::Granted request >Thread-46::DEBUG::2014-08-22 09:50:55,787::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`64cfb026-93d8-4d68-9aff-94391b27c183`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:50:55,787::task::993::Storage.TaskManager.Task::(_decref) Task=`64cfb026-93d8-4d68-9aff-94391b27c183`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:50:55,789::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:55,789::task::1191::Storage.TaskManager.Task::(prepare) Task=`64cfb026-93d8-4d68-9aff-94391b27c183`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:50:55,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`64cfb026-93d8-4d68-9aff-94391b27c183`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:50:55,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:50:55,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:50:55,790::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:50:55,790::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:50:55,790::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:50:55,790::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:50:55,790::task::993::Storage.TaskManager.Task::(_decref) Task=`64cfb026-93d8-4d68-9aff-94391b27c183`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:50:55,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:55,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011529 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:50:57,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:57,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116302 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:59,002::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:50:59,002::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:59,003::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:59,003::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:50:59,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:50:59,004::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:59,010::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:50:59,011::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196274 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:50:59,012::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) (1555) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) (1555) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) (1555) Building response >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) (1555) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:50:59,013::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) (1556) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:50:59,014::__init__::225::IOProcess::(_processLogs) (1556) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:50:59,015::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1556) Building response >Thread-17::DEBUG::2014-08-22 09:50:59,015::__init__::225::IOProcess::(_processLogs) (1556) Queuing response >Thread-17::DEBUG::2014-08-22 09:50:59,016::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:50:59,016::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:50:59,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:50:59,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188295 s, 54.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:00,047::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:00,047::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:00,047::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:00,047::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:00,047::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:00,047::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:00,048::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:00,048::__init__::225::IOProcess::(_processLogs) (1557) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:00,048::__init__::225::IOProcess::(_processLogs) (1557) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:00,048::__init__::225::IOProcess::(_processLogs) (1557) Building response >Thread-17::DEBUG::2014-08-22 09:51:00,048::__init__::225::IOProcess::(_processLogs) (1557) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:00,048::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:51:00,049::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:00,049::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:51:00,055::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000271496 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:00,056::__init__::225::IOProcess::(_processLogs) (1558) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:00,057::__init__::225::IOProcess::(_processLogs) (1558) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:00,057::__init__::225::IOProcess::(_processLogs) (1558) Building response >Thread-17::DEBUG::2014-08-22 09:51:00,057::__init__::225::IOProcess::(_processLogs) (1558) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:00,057::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:51:00,058::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:00,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:00,058::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:51:00,059::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:00,059::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:00,059::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:00,059::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) (1559) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) (1559) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) (1559) Building response >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) (1559) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:51:00,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) (1560) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) (1560) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:00,061::__init__::225::IOProcess::(_processLogs) (1560) Building response >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) (1560) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:00,062::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:00,063::__init__::225::IOProcess::(_processLogs) (1561) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:00,063::__init__::225::IOProcess::(_processLogs) (1561) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:00,063::__init__::225::IOProcess::(_processLogs) (1561) Building response >Thread-17::DEBUG::2014-08-22 09:51:00,063::__init__::225::IOProcess::(_processLogs) (1561) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:00,063::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:00,063::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:51:01,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:02,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216801 s, 47.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:04,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:04,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117095 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:51:05,840::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:51:05,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4ab38a4-d0f5-451a-89e5-65a8e6168797`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:05,840::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:51:05,840::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:05,840::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4ab38a4-d0f5-451a-89e5-65a8e6168797`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:05,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4ab38a4-d0f5-451a-89e5-65a8e6168797`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:05,841::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:05,841::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:05,841::task::993::Storage.TaskManager.Task::(_decref) Task=`c4ab38a4-d0f5-451a-89e5-65a8e6168797`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:51:05,874::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:51:05,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`febf0aad-9981-44d0-8045-af05d4f248e7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:51:05,874::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:51:05,875::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8f3c07c6-de91-416e-847b-d120d01053b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:51:05,875::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:51:05,875::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:51:05,875::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8f3c07c6-de91-416e-847b-d120d01053b3`::Granted request >Thread-46::DEBUG::2014-08-22 09:51:05,875::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`febf0aad-9981-44d0-8045-af05d4f248e7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:51:05,875::task::993::Storage.TaskManager.Task::(_decref) Task=`febf0aad-9981-44d0-8045-af05d4f248e7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:51:05,879::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:05,879::task::1191::Storage.TaskManager.Task::(prepare) Task=`febf0aad-9981-44d0-8045-af05d4f248e7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107473797120', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:05,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`febf0aad-9981-44d0-8045-af05d4f248e7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:51:05,879::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:51:05,879::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:51:05,880::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:51:05,880::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:51:05,880::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:51:05,880::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:51:05,880::task::993::Storage.TaskManager.Task::(_decref) Task=`febf0aad-9981-44d0-8045-af05d4f248e7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:06,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:06,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116842 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:08,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:08,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116711 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:09,016::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:09,016::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:09,016::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:09,016::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:09,016::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:09,017::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:09,017::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:09,017::__init__::225::IOProcess::(_processLogs) (1562) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:09,017::__init__::225::IOProcess::(_processLogs) (1562) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:09,017::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1562) Building response >Thread-21::DEBUG::2014-08-22 09:51:09,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:09,018::__init__::225::IOProcess::(_processLogs) (1562) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:09,024::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:51:09,025::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000179868 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:09,025::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:09,026::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) (1563) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) (1563) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) (1563) Building response >Thread-17::DEBUG::2014-08-22 09:51:09,027::__init__::225::IOProcess::(_processLogs) (1563) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:09,028::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:09,028::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:51:09,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`27f6075f-be4b-4251-8b4a-0fe8d71e2b02`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:09,844::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:51:09,844::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000179868', 'lastCheck': '0.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000271496', 'lastCheck': '9.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:09,844::task::1191::Storage.TaskManager.Task::(prepare) Task=`27f6075f-be4b-4251-8b4a-0fe8d71e2b02`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000179868', 'lastCheck': '0.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000271496', 'lastCheck': '9.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:09,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`27f6075f-be4b-4251-8b4a-0fe8d71e2b02`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:09,844::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:09,844::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:09,844::task::993::Storage.TaskManager.Task::(_decref) Task=`27f6075f-be4b-4251-8b4a-0fe8d71e2b02`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:51:10,064::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:10,064::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:10,064::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:10,064::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:10,064::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:10,064::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:10,065::__init__::225::IOProcess::(_processLogs) (1564) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:10,065::__init__::225::IOProcess::(_processLogs) (1564) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:10,065::__init__::225::IOProcess::(_processLogs) (1564) Queuing response >Thread-22::DEBUG::2014-08-22 09:51:10,066::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:10,066::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:10,072::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:51:10,073::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000177008 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) (1565) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:10,074::__init__::225::IOProcess::(_processLogs) (1565) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:10,075::__init__::225::IOProcess::(_processLogs) (1565) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:10,076::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:10,076::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:10,076::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:51:10,076::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:10,076::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) (1566) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) (1566) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) (1566) Building response >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) (1566) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:10,077::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) (1567) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) (1567) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:10,078::__init__::225::IOProcess::(_processLogs) (1567) Building response >Thread-17::DEBUG::2014-08-22 09:51:10,079::__init__::225::IOProcess::(_processLogs) (1567) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:10,079::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:10,079::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:10,079::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:51:10,079::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:10,079::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) (1568) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) (1568) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) (1568) Building response >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) (1568) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:10,080::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:10,081::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:51:10,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:10,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121885 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:12,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:12,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117408 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:14,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:14,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116163 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:51:15,935::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:51:15,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb73c603-6628-47d6-9331-35b728ecce9d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:15,936::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:51:15,936::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:15,936::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb73c603-6628-47d6-9331-35b728ecce9d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:15,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb73c603-6628-47d6-9331-35b728ecce9d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:15,936::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:15,936::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:15,936::task::993::Storage.TaskManager.Task::(_decref) Task=`eb73c603-6628-47d6-9331-35b728ecce9d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:51:15,977::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:51:15,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`d88b879d-e47d-41d8-8493-b3dc5efb8a0f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:51:15,978::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:51:15,978::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`477cc408-539a-4870-95a5-e9f9bcf2f8d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:51:15,978::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:51:15,978::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:51:15,978::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`477cc408-539a-4870-95a5-e9f9bcf2f8d1`::Granted request >Thread-46::DEBUG::2014-08-22 09:51:15,979::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d88b879d-e47d-41d8-8493-b3dc5efb8a0f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:51:15,979::task::993::Storage.TaskManager.Task::(_decref) Task=`d88b879d-e47d-41d8-8493-b3dc5efb8a0f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:51:15,981::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107429756928', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107429756928', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:15,981::task::1191::Storage.TaskManager.Task::(prepare) Task=`d88b879d-e47d-41d8-8493-b3dc5efb8a0f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107429756928', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107429756928', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:15,981::task::595::Storage.TaskManager.Task::(_updateState) Task=`d88b879d-e47d-41d8-8493-b3dc5efb8a0f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:51:15,981::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:51:15,981::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:51:15,981::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:51:15,981::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:51:15,981::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:51:15,981::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:51:15,981::task::993::Storage.TaskManager.Task::(_decref) Task=`d88b879d-e47d-41d8-8493-b3dc5efb8a0f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:16,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:16,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116124 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:18,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:18,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115516 s, 88.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) (1569) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:19,030::__init__::225::IOProcess::(_processLogs) (1569) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:19,031::__init__::225::IOProcess::(_processLogs) (1569) Queuing response >Thread-21::DEBUG::2014-08-22 09:51:19,031::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:19,032::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:19,038::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:51:19,039::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000199029 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:19,039::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:19,039::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:19,039::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:19,039::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:19,039::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:19,040::__init__::225::IOProcess::(_processLogs) (1570) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:19,040::__init__::225::IOProcess::(_processLogs) (1570) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:19,040::__init__::225::IOProcess::(_processLogs) (1570) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:19,041::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:19,041::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:20,081::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:20,082::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:20,083::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:20,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:20,084::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:20,087::__init__::225::IOProcess::(_processLogs) (1571) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 09:51:20,088::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:20,088::__init__::225::IOProcess::(_processLogs) (1571) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:20,095::__init__::225::IOProcess::(_processLogs) (1571) Building response >Thread-17::DEBUG::2014-08-22 09:51:20,097::__init__::225::IOProcess::(_processLogs) (1571) Queuing response >Thread-22::DEBUG::2014-08-22 09:51:20,097::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0002236 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:20,098::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:20,099::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:20,101::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:20,101::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:20,102::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:20,103::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:20,103::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:20,104::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:20,105::__init__::225::IOProcess::(_processLogs) (1572) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:20,105::__init__::225::IOProcess::(_processLogs) (1572) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:20,108::__init__::225::IOProcess::(_processLogs) (1572) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:20,109::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:20,110::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:20,111::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:51:20,112::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:20,112::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:20,113::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:20,114::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:20,115::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:20,115::__init__::225::IOProcess::(_processLogs) (1573) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:20,116::__init__::225::IOProcess::(_processLogs) (1573) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:20,117::__init__::225::IOProcess::(_processLogs) (1573) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:20,118::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:20,119::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:20,120::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:51:20,121::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:20,122::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:20,122::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:20,123::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:20,124::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:20,124::__init__::225::IOProcess::(_processLogs) (1574) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:20,125::__init__::225::IOProcess::(_processLogs) (1574) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:20,127::__init__::225::IOProcess::(_processLogs) (1574) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:20,127::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:20,128::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:20,130::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:51:20,131::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:20,131::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:20,132::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:20,133::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:20,133::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:20,134::__init__::225::IOProcess::(_processLogs) (1575) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:20,135::__init__::225::IOProcess::(_processLogs) (1575) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:20,136::__init__::225::IOProcess::(_processLogs) (1575) Building response >Thread-17::DEBUG::2014-08-22 09:51:20,136::__init__::225::IOProcess::(_processLogs) (1575) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:20,137::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-45::DEBUG::2014-08-22 09:51:20,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:20,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120887 s, 84.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:22,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:22,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119478 s, 85.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:24,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:24,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115895 s, 88.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:51:25,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c2d859e-f83f-4e76-9a03-8a3e54d22bab`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:25,071::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:51:25,071::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199029', 'lastCheck': '6.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002236', 'lastCheck': '4.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:25,071::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c2d859e-f83f-4e76-9a03-8a3e54d22bab`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000199029', 'lastCheck': '6.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002236', 'lastCheck': '4.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:25,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c2d859e-f83f-4e76-9a03-8a3e54d22bab`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:25,071::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:25,071::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:25,071::task::993::Storage.TaskManager.Task::(_decref) Task=`5c2d859e-f83f-4e76-9a03-8a3e54d22bab`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:51:26,046::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-47::DEBUG::2014-08-22 09:51:26,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`5010e539-ef9b-4b6d-9008-0e869409de4c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:26,047::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:51:26,047::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:26,047::task::1191::Storage.TaskManager.Task::(prepare) Task=`5010e539-ef9b-4b6d-9008-0e869409de4c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:26,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`5010e539-ef9b-4b6d-9008-0e869409de4c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:26,047::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:26,047::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:26,047::task::993::Storage.TaskManager.Task::(_decref) Task=`5010e539-ef9b-4b6d-9008-0e869409de4c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:51:26,101::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-46::DEBUG::2014-08-22 09:51:26,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`c39b0b03-ed9d-4750-afe1-f8b369aabaae`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:51:26,101::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:51:26,101::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`60b26507-6ea9-47f2-a732-2ddbfbd3d306`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:51:26,102::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:51:26,102::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:51:26,102::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`60b26507-6ea9-47f2-a732-2ddbfbd3d306`::Granted request >Thread-46::DEBUG::2014-08-22 09:51:26,102::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c39b0b03-ed9d-4750-afe1-f8b369aabaae`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:51:26,102::task::993::Storage.TaskManager.Task::(_decref) Task=`c39b0b03-ed9d-4750-afe1-f8b369aabaae`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:51:26,104::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107386765312', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107386765312', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:26,104::task::1191::Storage.TaskManager.Task::(prepare) Task=`c39b0b03-ed9d-4750-afe1-f8b369aabaae`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107386765312', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107386765312', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:26,104::task::595::Storage.TaskManager.Task::(_updateState) Task=`c39b0b03-ed9d-4750-afe1-f8b369aabaae`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:51:26,104::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:51:26,104::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:51:26,104::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:51:26,105::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:51:26,105::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:51:26,105::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:51:26,105::task::993::Storage.TaskManager.Task::(_decref) Task=`c39b0b03-ed9d-4750-afe1-f8b369aabaae`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:26,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:26,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164104 s, 62.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:28,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:28,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116063 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) (1576) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:29,042::__init__::225::IOProcess::(_processLogs) (1576) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:29,049::__init__::225::IOProcess::(_processLogs) (1576) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:29,050::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:51:29,051::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:51:29,057::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000204223 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:29,057::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:29,058::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:29,058::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:29,058::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:29,058::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:29,058::__init__::225::IOProcess::(_processLogs) (1577) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:29,058::__init__::225::IOProcess::(_processLogs) (1577) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:29,059::__init__::225::IOProcess::(_processLogs) (1577) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:29,060::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:29,060::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:30,139::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) (1578) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) (1578) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) (1578) Building response >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) (1578) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:30,140::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:51:30,141::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:51:30,148::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206618 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:30,151::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:30,151::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:30,152::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:30,153::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:30,153::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:30,156::__init__::225::IOProcess::(_processLogs) (1579) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:30,157::__init__::225::IOProcess::(_processLogs) (1579) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:30,159::__init__::225::IOProcess::(_processLogs) (1579) Building response >Thread-17::DEBUG::2014-08-22 09:51:30,160::__init__::225::IOProcess::(_processLogs) (1579) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:30,160::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:30,161::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:30,162::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:51:30,163::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:30,164::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:30,166::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:30,169::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:30,170::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:30,172::__init__::225::IOProcess::(_processLogs) (1580) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:30,173::__init__::225::IOProcess::(_processLogs) (1580) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:30,174::__init__::225::IOProcess::(_processLogs) (1580) Building response >Thread-17::DEBUG::2014-08-22 09:51:30,175::__init__::225::IOProcess::(_processLogs) (1580) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:30,175::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:30,176::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:30,177::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:51:30,177::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:30,178::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:30,178::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:30,179::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:30,180::__init__::225::IOProcess::(_processLogs) (1581) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:30,180::__init__::225::IOProcess::(_processLogs) (1581) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:30,181::__init__::225::IOProcess::(_processLogs) (1581) Building response >Thread-17::DEBUG::2014-08-22 09:51:30,181::__init__::225::IOProcess::(_processLogs) (1581) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:30,182::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:30,182::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:30,183::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:51:30,184::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:30,184::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:30,185::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:30,185::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:30,186::__init__::225::IOProcess::(_processLogs) (1582) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:30,188::__init__::225::IOProcess::(_processLogs) (1582) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:30,188::__init__::225::IOProcess::(_processLogs) (1582) Building response >Thread-17::DEBUG::2014-08-22 09:51:30,188::__init__::225::IOProcess::(_processLogs) (1582) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:30,189::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:30,189::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:51:30,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:30,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121164 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:32,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:32,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220957 s, 46.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:34,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:35,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119792 s, 85.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:51:36,208::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:51:36,208::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5631da8-ebbf-47d3-bf93-786e7e2d5bde`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:36,208::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:51:36,209::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:36,209::task::1191::Storage.TaskManager.Task::(prepare) Task=`e5631da8-ebbf-47d3-bf93-786e7e2d5bde`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:36,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5631da8-ebbf-47d3-bf93-786e7e2d5bde`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:36,209::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:36,209::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:36,209::task::993::Storage.TaskManager.Task::(_decref) Task=`e5631da8-ebbf-47d3-bf93-786e7e2d5bde`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:51:36,247::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:51:36,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`8630f3fe-d63a-4a4d-bab3-477e0d67edbd`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:51:36,247::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:51:36,247::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f827d421-dccc-49de-a9a1-05b68f5afb8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:51:36,247::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:51:36,247::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:51:36,247::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f827d421-dccc-49de-a9a1-05b68f5afb8b`::Granted request >Thread-46::DEBUG::2014-08-22 09:51:36,248::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8630f3fe-d63a-4a4d-bab3-477e0d67edbd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:51:36,248::task::993::Storage.TaskManager.Task::(_decref) Task=`8630f3fe-d63a-4a4d-bab3-477e0d67edbd`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:51:36,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:36,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`8630f3fe-d63a-4a4d-bab3-477e0d67edbd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:36,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`8630f3fe-d63a-4a4d-bab3-477e0d67edbd`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:51:36,250::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:51:36,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:51:36,250::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:51:36,250::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:51:36,250::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:51:36,251::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:51:36,251::task::993::Storage.TaskManager.Task::(_decref) Task=`8630f3fe-d63a-4a4d-bab3-477e0d67edbd`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:37,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:37,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131108 s, 78.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:39,060::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:39,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:39,060::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:39,060::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:39,061::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:39,061::__init__::225::IOProcess::(_processLogs) (1583) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:39,061::__init__::225::IOProcess::(_processLogs) (1583) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:39,061::__init__::225::IOProcess::(_processLogs) (1583) Queuing response >Thread-21::DEBUG::2014-08-22 09:51:39,062::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:39,062::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:39,068::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:51:39,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000268154 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:39,069::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) (1584) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:39,070::__init__::225::IOProcess::(_processLogs) (1584) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:39,071::__init__::225::IOProcess::(_processLogs) DEBUG|(1584) Building response >Thread-17::DEBUG::2014-08-22 09:51:39,072::__init__::225::IOProcess::(_processLogs) (1584) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:39,072::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:39,072::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:51:39,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:39,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114717 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:40,188::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:40,188::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:40,188::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:40,189::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:51:40,190::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:40,190::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:40,196::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:40,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:40,196::__init__::225::IOProcess::(_processLogs) (1585) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:40,196::__init__::225::IOProcess::(_processLogs) (1585) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:40,196::__init__::225::IOProcess::(_processLogs) (1585) Building response >Thread-17::DEBUG::2014-08-22 09:51:40,197::__init__::225::IOProcess::(_processLogs) (1585) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:40,197::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:40,197::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:51:40,197::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000805074 s, 708 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:40,198::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:40,198::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:40,198::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:40,199::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:40,199::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:40,199::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:40,199::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:40,199::__init__::225::IOProcess::(_processLogs) (1586) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:40,199::__init__::225::IOProcess::(_processLogs) (1586) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) (1586) Building response >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) (1586) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:40,200::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) (1587) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) (1587) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) (1587) Building response >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) (1587) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:40,201::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:40,202::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:40,202::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:40,202::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:51:40,202::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:40,202::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) (1588) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) (1588) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) (1588) Building response >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) (1588) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:40,203::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:51:40,204::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:40,204::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:40,204::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) (1589) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) (1589) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) (1589) Building response >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) (1589) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:40,205::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:51:40,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c0c197c-7b6f-44eb-bd34-77a8172dfc9c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:40,256::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:51:40,256::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000268154', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000805074', 'lastCheck': '0.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:40,256::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c0c197c-7b6f-44eb-bd34-77a8172dfc9c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000268154', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000805074', 'lastCheck': '0.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:40,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c0c197c-7b6f-44eb-bd34-77a8172dfc9c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:40,256::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:40,256::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:40,256::task::993::Storage.TaskManager.Task::(_decref) Task=`8c0c197c-7b6f-44eb-bd34-77a8172dfc9c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:41,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:41,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158596 s, 64.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:43,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:43,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116026 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:45,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:45,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116286 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:51:46,328::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:51:46,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae6fca0e-48b6-4324-b1fa-dd8e835b638e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:46,328::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:51:46,328::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:46,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`ae6fca0e-48b6-4324-b1fa-dd8e835b638e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:46,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae6fca0e-48b6-4324-b1fa-dd8e835b638e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:46,329::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:46,329::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:46,329::task::993::Storage.TaskManager.Task::(_decref) Task=`ae6fca0e-48b6-4324-b1fa-dd8e835b638e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:51:46,396::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:51:46,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec9fbfc9-ecef-461e-8486-995356237d6d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:51:46,396::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:51:46,397::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8958d626-9ebb-4934-bcad-19fffb69d58d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:51:46,397::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:51:46,397::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:51:46,397::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8958d626-9ebb-4934-bcad-19fffb69d58d`::Granted request >Thread-46::DEBUG::2014-08-22 09:51:46,397::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ec9fbfc9-ecef-461e-8486-995356237d6d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:51:46,397::task::993::Storage.TaskManager.Task::(_decref) Task=`ec9fbfc9-ecef-461e-8486-995356237d6d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:51:46,399::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:46,399::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec9fbfc9-ecef-461e-8486-995356237d6d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:46,399::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec9fbfc9-ecef-461e-8486-995356237d6d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:51:46,400::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:51:46,400::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:51:46,400::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:51:46,400::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:51:46,400::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:51:46,400::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:51:46,400::task::993::Storage.TaskManager.Task::(_decref) Task=`ec9fbfc9-ecef-461e-8486-995356237d6d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:47,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:47,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119108 s, 86.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:49,072::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:49,072::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:49,072::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:49,072::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:49,072::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:49,073::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:49,073::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:49,073::__init__::225::IOProcess::(_processLogs) (1590) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:49,073::__init__::225::IOProcess::(_processLogs) (1590) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:49,073::__init__::225::IOProcess::(_processLogs) DEBUG|(1590) Building response >Thread-17::DEBUG::2014-08-22 09:51:49,074::__init__::225::IOProcess::(_processLogs) (1590) Queuing response >Thread-21::DEBUG::2014-08-22 09:51:49,074::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:49,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:49,081::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:51:49,081::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000188701 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:49,082::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:49,083::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:49,083::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:49,083::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:49,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:49,084::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:49,084::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:49,084::__init__::225::IOProcess::(_processLogs) (1591) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:49,085::__init__::225::IOProcess::(_processLogs) (1591) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:49,085::__init__::225::IOProcess::(_processLogs) (1591) Building response >Thread-17::DEBUG::2014-08-22 09:51:49,085::__init__::225::IOProcess::(_processLogs) (1591) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:49,085::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:49,085::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:51:49,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:49,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119286 s, 85.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:50,205::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:50,205::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:50,205::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:50,206::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:51:50,207::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:50,207::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:50,213::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:51:50,214::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234363 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:50,214::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:50,215::__init__::225::IOProcess::(_processLogs) (1592) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:50,215::__init__::225::IOProcess::(_processLogs) (1592) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:50,215::__init__::225::IOProcess::(_processLogs) (1592) Building response >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) (1592) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:50,216::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:50,217::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:50,217::__init__::225::IOProcess::(_processLogs) (1593) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:50,217::__init__::225::IOProcess::(_processLogs) (1593) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:50,217::__init__::225::IOProcess::(_processLogs) (1593) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:50,218::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:50,218::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:51:50,218::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:51:50,218::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) (1594) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) (1594) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) (1594) Building response >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) (1594) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:50,219::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:50,220::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:50,220::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:51:50,220::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:50,220::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:50,220::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) (1595) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) (1595) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) (1595) Building response >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) (1595) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:51:50,221::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:50,222::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:50,222::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:50,222::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:50,222::__init__::225::IOProcess::(_processLogs) (1596) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:51:50,223::__init__::225::IOProcess::(_processLogs) (1596) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:51:50,223::__init__::225::IOProcess::(_processLogs) (1596) Building response >Thread-17::DEBUG::2014-08-22 09:51:50,223::__init__::225::IOProcess::(_processLogs) (1596) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:50,223::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:50,223::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:51:51,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:51,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119886 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:53,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:53,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118899 s, 86.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:51:55,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:55,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181754 s, 56.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:51:56,093::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9d81b8e-d347-4a55-8313-1da04121ad89`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:56,093::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:51:56,093::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000188701', 'lastCheck': '7.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234363', 'lastCheck': '5.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:56,093::task::1191::Storage.TaskManager.Task::(prepare) Task=`f9d81b8e-d347-4a55-8313-1da04121ad89`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000188701', 'lastCheck': '7.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234363', 'lastCheck': '5.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:51:56,093::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9d81b8e-d347-4a55-8313-1da04121ad89`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:56,093::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:56,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:56,094::task::993::Storage.TaskManager.Task::(_decref) Task=`f9d81b8e-d347-4a55-8313-1da04121ad89`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:51:56,467::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:51:56,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`e593948c-6238-4e1b-8769-6c03f0b25a40`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:51:56,467::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:51:56,467::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:56,467::task::1191::Storage.TaskManager.Task::(prepare) Task=`e593948c-6238-4e1b-8769-6c03f0b25a40`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:51:56,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`e593948c-6238-4e1b-8769-6c03f0b25a40`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:51:56,468::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:51:56,468::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:51:56,468::task::993::Storage.TaskManager.Task::(_decref) Task=`e593948c-6238-4e1b-8769-6c03f0b25a40`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:51:56,500::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:51:56,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb32d5b-d64e-473f-8172-1b931329defe`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:51:56,501::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:51:56,501::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`29ac1ad7-ee7e-49a7-a04a-1adaf2abb4f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:51:56,501::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:51:56,501::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:51:56,501::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`29ac1ad7-ee7e-49a7-a04a-1adaf2abb4f5`::Granted request >Thread-46::DEBUG::2014-08-22 09:51:56,501::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`beb32d5b-d64e-473f-8172-1b931329defe`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:51:56,502::task::993::Storage.TaskManager.Task::(_decref) Task=`beb32d5b-d64e-473f-8172-1b931329defe`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:51:56,503::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:56,504::task::1191::Storage.TaskManager.Task::(prepare) Task=`beb32d5b-d64e-473f-8172-1b931329defe`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:51:56,504::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb32d5b-d64e-473f-8172-1b931329defe`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:51:56,504::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:51:56,504::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:51:56,504::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:51:56,504::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:51:56,504::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:51:56,504::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:51:56,504::task::993::Storage.TaskManager.Task::(_decref) Task=`beb32d5b-d64e-473f-8172-1b931329defe`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:51:57,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:57,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115722 s, 88.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) (1597) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:59,085::__init__::225::IOProcess::(_processLogs) (1597) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:59,086::__init__::225::IOProcess::(_processLogs) (1597) Queuing response >Thread-21::DEBUG::2014-08-22 09:51:59,087::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:51:59,087::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:51:59,093::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:51:59,094::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00019535 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:51:59,094::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:51:59,095::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:51:59,095::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:51:59,095::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:51:59,095::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:51:59,095::__init__::225::IOProcess::(_processLogs) (1598) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:51:59,095::__init__::225::IOProcess::(_processLogs) (1598) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:51:59,096::__init__::225::IOProcess::(_processLogs) (1598) Queuing response >Thread-17::DEBUG::2014-08-22 09:51:59,096::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:51:59,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:51:59,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154987 s, 66.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) (1599) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:00,223::__init__::225::IOProcess::(_processLogs) (1599) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:00,224::__init__::225::IOProcess::(_processLogs) (1599) Queuing response >Thread-22::DEBUG::2014-08-22 09:52:00,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:00,225::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:00,230::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:52:00,233::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00151901 s, 375 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) (1600) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:00,234::__init__::225::IOProcess::(_processLogs) (1600) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:00,235::__init__::225::IOProcess::(_processLogs) (1600) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:00,235::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:00,236::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:52:00,236::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:00,236::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:00,237::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:00,237::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:00,237::__init__::225::IOProcess::(_processLogs) (1601) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:00,237::__init__::225::IOProcess::(_processLogs) (1601) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:00,237::__init__::225::IOProcess::(_processLogs) (1601) Building response >Thread-17::DEBUG::2014-08-22 09:52:00,237::__init__::225::IOProcess::(_processLogs) (1601) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) (1602) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:00,238::__init__::225::IOProcess::(_processLogs) (1602) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:00,239::__init__::225::IOProcess::(_processLogs) (1602) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:00,239::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:00,239::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:00,239::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:52:00,239::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) (1603) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) (1603) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) (1603) Building response >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) (1603) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:00,240::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-45::DEBUG::2014-08-22 09:52:01,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:01,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119592 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:03,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:03,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129696 s, 79.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:05,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:05,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116825 s, 87.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:06,696::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:52:06,696::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7ec0cfb-49e6-4412-a595-8de89c7f7807`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:06,697::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:52:06,697::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:06,697::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7ec0cfb-49e6-4412-a595-8de89c7f7807`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:06,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7ec0cfb-49e6-4412-a595-8de89c7f7807`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:06,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:06,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:06,698::task::993::Storage.TaskManager.Task::(_decref) Task=`c7ec0cfb-49e6-4412-a595-8de89c7f7807`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:52:06,721::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:52:06,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab98c14-5e6d-4a43-a19a-b916e8aea077`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:52:06,721::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:52:06,722::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e899312b-e9f8-4c16-9ad0-5b146c32039b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:52:06,722::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:52:06,722::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:52:06,722::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e899312b-e9f8-4c16-9ad0-5b146c32039b`::Granted request >Thread-46::DEBUG::2014-08-22 09:52:06,722::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bab98c14-5e6d-4a43-a19a-b916e8aea077`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:52:06,722::task::993::Storage.TaskManager.Task::(_decref) Task=`bab98c14-5e6d-4a43-a19a-b916e8aea077`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:52:06,724::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:06,724::task::1191::Storage.TaskManager.Task::(prepare) Task=`bab98c14-5e6d-4a43-a19a-b916e8aea077`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:06,724::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab98c14-5e6d-4a43-a19a-b916e8aea077`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:52:06,724::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:52:06,725::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:52:06,725::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:52:06,725::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:52:06,725::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:52:06,725::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:52:06,725::task::993::Storage.TaskManager.Task::(_decref) Task=`bab98c14-5e6d-4a43-a19a-b916e8aea077`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:07,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:07,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115163 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) (1604) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:09,098::__init__::225::IOProcess::(_processLogs) (1604) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:09,099::__init__::225::IOProcess::(_processLogs) (1604) Queuing response >Thread-21::DEBUG::2014-08-22 09:52:09,099::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:09,100::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:09,106::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:52:09,107::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000190827 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:09,107::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:09,108::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:09,108::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:09,108::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:09,108::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:09,108::__init__::225::IOProcess::(_processLogs) (1605) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:09,108::__init__::225::IOProcess::(_processLogs) (1605) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:09,109::__init__::225::IOProcess::(_processLogs) (1605) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:09,109::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:09,109::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:52:09,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:09,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119854 s, 85.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:10,241::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:10,241::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:10,241::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:10,242::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:10,242::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:10,242::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:10,242::__init__::225::IOProcess::(_processLogs) (1606) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:10,242::__init__::225::IOProcess::(_processLogs) (1606) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:10,242::__init__::225::IOProcess::(_processLogs) DEBUG|(1606) Building response >Thread-22::DEBUG::2014-08-22 09:52:10,243::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:10,243::__init__::225::IOProcess::(_processLogs) (1606) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:10,250::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:52:10,250::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000185369 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:10,251::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:10,252::__init__::225::IOProcess::(_processLogs) (1607) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:10,253::__init__::225::IOProcess::(_processLogs) (1607) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:10,253::__init__::225::IOProcess::(_processLogs) DEBUG|(1607) Building response >Thread-17::DEBUG::2014-08-22 09:52:10,254::__init__::225::IOProcess::(_processLogs) (1607) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:10,254::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:10,254::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:10,254::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:10,254::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:52:10,254::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) (1608) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) (1608) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) (1608) Building response >Thread-17::DEBUG::2014-08-22 09:52:10,255::__init__::225::IOProcess::(_processLogs) (1608) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:10,256::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:10,256::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:10,256::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:10,256::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:52:10,256::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:10,256::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) (1609) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) (1609) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) (1609) Building response >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) (1609) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:10,257::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:10,258::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:52:10,258::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:10,258::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:10,258::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) (1610) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) (1610) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) (1610) Building response >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) (1610) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:10,259::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:52:11,337::task::595::Storage.TaskManager.Task::(_updateState) Task=`601b77b0-496d-45e4-8909-100abc5d621d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:11,338::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:52:11,338::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000190827', 'lastCheck': '2.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185369', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:11,338::task::1191::Storage.TaskManager.Task::(prepare) Task=`601b77b0-496d-45e4-8909-100abc5d621d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000190827', 'lastCheck': '2.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185369', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:11,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`601b77b0-496d-45e4-8909-100abc5d621d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:11,338::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:11,338::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:11,338::task::993::Storage.TaskManager.Task::(_decref) Task=`601b77b0-496d-45e4-8909-100abc5d621d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:11,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:11,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119151 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:13,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:13,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116124 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:16,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:16,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166687 s, 61.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:16,806::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:52:16,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`c63a900d-2990-4a7e-be44-560a68c47248`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:16,807::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:52:16,807::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:16,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`c63a900d-2990-4a7e-be44-560a68c47248`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:16,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`c63a900d-2990-4a7e-be44-560a68c47248`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:16,807::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:16,808::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:16,808::task::993::Storage.TaskManager.Task::(_decref) Task=`c63a900d-2990-4a7e-be44-560a68c47248`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:52:16,842::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:52:16,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`79827049-f7e6-41a3-a8ed-e4930aea6f25`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:52:16,843::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:52:16,843::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cd7d8969-3069-4637-be4f-3d15246bb308`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:52:16,843::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:52:16,843::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:52:16,843::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cd7d8969-3069-4637-be4f-3d15246bb308`::Granted request >Thread-46::DEBUG::2014-08-22 09:52:16,843::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`79827049-f7e6-41a3-a8ed-e4930aea6f25`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:52:16,843::task::993::Storage.TaskManager.Task::(_decref) Task=`79827049-f7e6-41a3-a8ed-e4930aea6f25`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:52:16,845::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:16,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`79827049-f7e6-41a3-a8ed-e4930aea6f25`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:16,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`79827049-f7e6-41a3-a8ed-e4930aea6f25`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:52:16,846::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:52:16,846::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:52:16,846::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:52:16,846::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:52:16,846::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:52:16,846::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:52:16,846::task::993::Storage.TaskManager.Task::(_decref) Task=`79827049-f7e6-41a3-a8ed-e4930aea6f25`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:18,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:18,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116536 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:19,110::__init__::225::IOProcess::(_processLogs) (1611) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:19,111::__init__::225::IOProcess::(_processLogs) (1611) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:19,111::__init__::225::IOProcess::(_processLogs) (1611) Queuing response >Thread-21::DEBUG::2014-08-22 09:52:19,112::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:19,112::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:19,118::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:52:19,119::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000195416 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) (1612) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:19,120::__init__::225::IOProcess::(_processLogs) (1612) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:19,121::__init__::225::IOProcess::(_processLogs) (1612) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:19,122::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:19,122::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:52:20,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:20,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117462 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:20,259::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:20,259::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:20,259::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:20,259::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:20,260::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:20,260::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:20,260::__init__::225::IOProcess::(_processLogs) (1613) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:20,260::__init__::225::IOProcess::(_processLogs) (1613) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:20,260::__init__::225::IOProcess::(_processLogs) (1613) Queuing response >Thread-22::DEBUG::2014-08-22 09:52:20,261::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:20,261::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:20,267::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:52:20,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000226334 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:20,269::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:20,270::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:20,270::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:20,270::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:20,270::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:20,270::__init__::225::IOProcess::(_processLogs) (1614) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:20,270::__init__::225::IOProcess::(_processLogs) (1614) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:20,271::__init__::225::IOProcess::(_processLogs) (1614) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:20,271::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:20,271::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) (1615) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) (1615) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) (1615) Building response >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) (1615) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:20,272::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:20,273::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:52:20,273::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:20,273::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) (1616) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) (1616) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) (1616) Building response >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) (1616) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:52:20,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:20,275::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:20,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) (1617) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) (1617) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) (1617) Building response >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) (1617) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:20,276::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:52:22,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:22,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118095 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:24,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:24,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264163 s, 38.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:26,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:26,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117873 s, 86.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:26,537::task::595::Storage.TaskManager.Task::(_updateState) Task=`99d490fb-8cc3-4b43-b377-13db456b0343`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:26,537::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:52:26,537::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000195416', 'lastCheck': '7.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000226334', 'lastCheck': '6.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:26,537::task::1191::Storage.TaskManager.Task::(prepare) Task=`99d490fb-8cc3-4b43-b377-13db456b0343`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000195416', 'lastCheck': '7.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000226334', 'lastCheck': '6.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:26,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`99d490fb-8cc3-4b43-b377-13db456b0343`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:26,538::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:26,538::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:26,538::task::993::Storage.TaskManager.Task::(_decref) Task=`99d490fb-8cc3-4b43-b377-13db456b0343`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:52:26,894::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:52:26,895::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb72ecd7-3f4b-4d99-a0ac-2ede02d606ef`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:26,895::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:52:26,895::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:26,895::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb72ecd7-3f4b-4d99-a0ac-2ede02d606ef`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:26,895::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb72ecd7-3f4b-4d99-a0ac-2ede02d606ef`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:26,895::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:26,895::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:26,895::task::993::Storage.TaskManager.Task::(_decref) Task=`bb72ecd7-3f4b-4d99-a0ac-2ede02d606ef`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:52:26,921::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:52:26,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1a1f857-a970-4af2-a163-b2ba5b2dbad7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:52:26,921::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:52:26,921::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`11ccbdf9-a915-45ac-8c6b-182d4f135f8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:52:26,921::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:52:26,922::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:52:26,922::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`11ccbdf9-a915-45ac-8c6b-182d4f135f8c`::Granted request >Thread-46::DEBUG::2014-08-22 09:52:26,922::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e1a1f857-a970-4af2-a163-b2ba5b2dbad7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:52:26,922::task::993::Storage.TaskManager.Task::(_decref) Task=`e1a1f857-a970-4af2-a163-b2ba5b2dbad7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:52:26,924::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:26,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1a1f857-a970-4af2-a163-b2ba5b2dbad7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:26,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1a1f857-a970-4af2-a163-b2ba5b2dbad7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:52:26,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:52:26,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:52:26,924::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:52:26,924::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:52:26,925::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:52:26,925::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:52:26,925::task::993::Storage.TaskManager.Task::(_decref) Task=`e1a1f857-a970-4af2-a163-b2ba5b2dbad7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:28,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:28,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116644 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:29,122::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:29,122::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:29,123::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:52:29,124::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:29,124::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:29,130::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:52:29,131::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219926 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:29,131::__init__::225::IOProcess::(_processLogs) (1618) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:29,132::__init__::225::IOProcess::(_processLogs) (1618) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:29,132::__init__::225::IOProcess::(_processLogs) (1618) Building response >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) (1618) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:29,133::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:29,134::__init__::225::IOProcess::(_processLogs) (1619) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:29,134::__init__::225::IOProcess::(_processLogs) (1619) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:29,134::__init__::225::IOProcess::(_processLogs) (1619) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:29,135::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:29,135::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:30,276::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:30,276::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:30,276::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:30,276::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:30,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:30,277::__init__::225::IOProcess::(_processLogs) (1620) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:30,277::__init__::225::IOProcess::(_processLogs) (1620) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:30,277::__init__::225::IOProcess::(_processLogs) (1620) Queuing response >Thread-22::DEBUG::2014-08-22 09:52:30,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:30,278::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:30,285::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:52:30,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000224315 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:30,286::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:30,286::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:30,286::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:30,286::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:30,287::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) (1621) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) (1621) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) (1621) Building response >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) (1621) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:52:30,288::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:30,289::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:30,289::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:30,289::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:30,289::__init__::225::IOProcess::(_processLogs) (1622) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:30,289::__init__::225::IOProcess::(_processLogs) (1622) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:30,289::__init__::225::IOProcess::(_processLogs) (1622) Building response >Thread-17::DEBUG::2014-08-22 09:52:30,290::__init__::225::IOProcess::(_processLogs) (1622) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:30,290::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:30,290::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:30,290::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:52:30,290::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) (1623) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) (1623) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) (1623) Building response >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) (1623) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:30,291::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:30,292::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:52:30,292::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:30,292::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:30,292::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) (1624) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) (1624) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) (1624) Building response >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) (1624) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:30,293::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:52:30,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:30,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120695 s, 84.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:32,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:32,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212984 s, 48.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:34,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:34,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135504 s, 75.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:36,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:36,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011698 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:36,989::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:52:36,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c5ee86b-66c7-4cb8-b922-5b055f90d78a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:36,990::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:52:36,990::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:36,990::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c5ee86b-66c7-4cb8-b922-5b055f90d78a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:36,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c5ee86b-66c7-4cb8-b922-5b055f90d78a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:36,990::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:36,990::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:36,991::task::993::Storage.TaskManager.Task::(_decref) Task=`5c5ee86b-66c7-4cb8-b922-5b055f90d78a`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:52:37,018::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:52:37,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`378d2d72-fc83-4141-adc7-54f52a7a353d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:52:37,018::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:52:37,018::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e8ff209d-1069-4348-925d-c9c1a32644c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:52:37,019::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:52:37,019::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:52:37,019::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e8ff209d-1069-4348-925d-c9c1a32644c5`::Granted request >Thread-46::DEBUG::2014-08-22 09:52:37,019::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`378d2d72-fc83-4141-adc7-54f52a7a353d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:52:37,019::task::993::Storage.TaskManager.Task::(_decref) Task=`378d2d72-fc83-4141-adc7-54f52a7a353d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:52:37,021::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:37,021::task::1191::Storage.TaskManager.Task::(prepare) Task=`378d2d72-fc83-4141-adc7-54f52a7a353d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:37,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`378d2d72-fc83-4141-adc7-54f52a7a353d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:52:37,021::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:52:37,021::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:52:37,022::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:52:37,022::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:52:37,022::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:52:37,022::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:52:37,022::task::993::Storage.TaskManager.Task::(_decref) Task=`378d2d72-fc83-4141-adc7-54f52a7a353d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:38,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:38,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114312 s, 89.6 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:52:39,135::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:52:39,137::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:52:39,137::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,137::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,137::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,137::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,137::__init__::225::IOProcess::(_processLogs) (1625) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:52:39,138::__init__::225::IOProcess::(_processLogs) (1625) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 09:52:39,141::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:52:39,138::__init__::225::IOProcess::(_processLogs) (1625) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:39,141::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,141::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:52:39,141::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:39,141::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) (1626) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) (1626) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:52:39,142::__init__::225::IOProcess::(_processLogs) (1626) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,144::__init__::225::IOProcess::(_processLogs) (1627) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:52:39,144::__init__::225::IOProcess::(_processLogs) (1627) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:52:39,168::__init__::225::IOProcess::(_processLogs) (1627) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:39,168::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,168::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-21::DEBUG::2014-08-22 09:52:39,168::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) (1628) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) (1628) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:52:39,169::__init__::225::IOProcess::(_processLogs) (1628) Queuing response >Thread-21::DEBUG::2014-08-22 09:52:39,170::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:52:39,170::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,171::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 09:52:39,172::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 09:52:39,172::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,172::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,172::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,172::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,172::__init__::225::IOProcess::(_processLogs) (1629) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) (1629) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) (1629) Building response >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) (1629) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,173::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,174::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,174::__init__::225::IOProcess::(_processLogs) (1630) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:52:39,174::__init__::225::IOProcess::(_processLogs) (1630) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 09:52:39,174::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:52:39,174::__init__::225::IOProcess::(_processLogs) (1630) Queuing response >Thread-21::INFO::2014-08-22 09:52:39,175::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:52:39,175::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 09:52:39,175::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:52:39,176::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 09:52:39,176::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:39,176::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:39,176::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,177::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,177::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,177::__init__::225::IOProcess::(_processLogs) (1631) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:39,177::__init__::225::IOProcess::(_processLogs) (1631) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:39,177::__init__::225::IOProcess::(_processLogs) (1631) Queuing response >Thread-21::DEBUG::2014-08-22 09:52:39,178::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:39,178::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,184::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:52:39,185::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000194477 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) (1632) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:39,186::__init__::225::IOProcess::(_processLogs) (1632) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:39,187::__init__::225::IOProcess::(_processLogs) (1632) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:39,188::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:39,188::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:52:40,293::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:52:40,294::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:52:40,295::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,295::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:40,295::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,295::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,295::__init__::225::IOProcess::(_processLogs) (1633) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) (1633) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) (1633) Building response >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) (1633) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,296::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,297::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,297::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,297::__init__::225::IOProcess::(_processLogs) (1634) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:52:40,297::__init__::225::IOProcess::(_processLogs) (1634) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:52:40,297::__init__::225::IOProcess::(_processLogs) (1634) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,298::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,299::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 09:52:40,300::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) (1635) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:52:40,301::__init__::225::IOProcess::(_processLogs) (1635) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:52:40,327::__init__::225::IOProcess::(_processLogs) (1635) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,327::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-22::DEBUG::2014-08-22 09:52:40,327::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) (1636) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:52:40,328::__init__::225::IOProcess::(_processLogs) (1636) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:52:40,329::__init__::225::IOProcess::(_processLogs) (1636) Queuing response >Thread-22::DEBUG::2014-08-22 09:52:40,329::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:52:40,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,330::__init__::225::IOProcess::(_processLogs) Sending response sized 832 >Thread-17::DEBUG::2014-08-22 09:52:40,330::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 09:52:40,331::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,331::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,331::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,331::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,331::__init__::225::IOProcess::(_processLogs) (1637) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:40,331::__init__::225::IOProcess::(_processLogs) (1637) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) (1637) Building response >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) (1637) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,332::__init__::225::IOProcess::(_processLogs) (1638) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:52:40,333::__init__::225::IOProcess::(_processLogs) (1638) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:52:40,333::__init__::225::IOProcess::(_processLogs) (1638) Queuing response >Thread-22::DEBUG::2014-08-22 09:52:40,333::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:52:40,334::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 09:52:40,334::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:52:40,334::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-22::INFO::2014-08-22 09:52:40,334::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) (1639) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:40,335::__init__::225::IOProcess::(_processLogs) (1639) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:40,336::__init__::225::IOProcess::(_processLogs) (1639) Queuing response >Thread-22::DEBUG::2014-08-22 09:52:40,337::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:40,337::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,343::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:52:40,344::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000183138 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:40,344::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:40,345::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:40,345::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,345::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,345::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,345::__init__::225::IOProcess::(_processLogs) (1640) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:40,345::__init__::225::IOProcess::(_processLogs) (1640) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:40,346::__init__::225::IOProcess::(_processLogs) (1640) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,347::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,347::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:40,347::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:52:40,347::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,347::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) (1641) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) (1641) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) (1641) Building response >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) (1641) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:40,348::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:52:40,349::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,349::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,349::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,349::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,349::__init__::225::IOProcess::(_processLogs) (1642) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:40,349::__init__::225::IOProcess::(_processLogs) (1642) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) (1642) Building response >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) (1642) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:40,350::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:40,351::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:40,351::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:40,351::__init__::225::IOProcess::(_processLogs) (1643) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:40,351::__init__::225::IOProcess::(_processLogs) (1643) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:40,351::__init__::225::IOProcess::(_processLogs) (1643) Building response >Thread-17::DEBUG::2014-08-22 09:52:40,352::__init__::225::IOProcess::(_processLogs) (1643) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:40,352::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:40,352::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:52:40,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:40,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121191 s, 84.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:41,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`8408da94-85d0-405d-bb26-9ffbe63bf31f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:41,730::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:52:41,730::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000194477', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000183138', 'lastCheck': '1.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:41,730::task::1191::Storage.TaskManager.Task::(prepare) Task=`8408da94-85d0-405d-bb26-9ffbe63bf31f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000194477', 'lastCheck': '2.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000183138', 'lastCheck': '1.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:41,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`8408da94-85d0-405d-bb26-9ffbe63bf31f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:41,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:41,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:41,730::task::993::Storage.TaskManager.Task::(_decref) Task=`8408da94-85d0-405d-bb26-9ffbe63bf31f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:42,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:42,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011705 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:44,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:44,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115267 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:46,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:46,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115706 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:47,070::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:52:47,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`99be7eed-c52c-4264-8813-ed79f6bea52c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:47,070::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:52:47,070::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:47,070::task::1191::Storage.TaskManager.Task::(prepare) Task=`99be7eed-c52c-4264-8813-ed79f6bea52c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:47,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`99be7eed-c52c-4264-8813-ed79f6bea52c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:47,071::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:47,071::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:47,071::task::993::Storage.TaskManager.Task::(_decref) Task=`99be7eed-c52c-4264-8813-ed79f6bea52c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:52:47,109::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:52:47,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`64c593be-ccfa-4384-880e-47e94b35082f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:52:47,109::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:52:47,109::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`eb9cf7b2-082b-41e4-ab7e-69c8529009e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:52:47,110::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:52:47,110::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:52:47,110::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`eb9cf7b2-082b-41e4-ab7e-69c8529009e8`::Granted request >Thread-46::DEBUG::2014-08-22 09:52:47,110::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`64c593be-ccfa-4384-880e-47e94b35082f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:52:47,110::task::993::Storage.TaskManager.Task::(_decref) Task=`64c593be-ccfa-4384-880e-47e94b35082f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:52:47,112::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:47,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`64c593be-ccfa-4384-880e-47e94b35082f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:47,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`64c593be-ccfa-4384-880e-47e94b35082f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:52:47,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:52:47,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:52:47,113::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:52:47,113::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:52:47,113::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:52:47,113::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:52:47,113::task::993::Storage.TaskManager.Task::(_decref) Task=`64c593be-ccfa-4384-880e-47e94b35082f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:48,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:49,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116342 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) (1644) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:49,189::__init__::225::IOProcess::(_processLogs) (1644) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:49,190::__init__::225::IOProcess::(_processLogs) (1644) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:49,191::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:52:49,191::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:49,191::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:52:49,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000180022 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:49,198::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:49,198::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:49,198::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:49,199::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:49,199::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:49,199::__init__::225::IOProcess::(_processLogs) (1645) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:49,199::__init__::225::IOProcess::(_processLogs) (1645) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:49,199::__init__::225::IOProcess::(_processLogs) (1645) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:49,200::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:49,200::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) (1646) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) (1646) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) (1646) Building response >Thread-17::DEBUG::2014-08-22 09:52:50,352::__init__::225::IOProcess::(_processLogs) (1646) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:50,353::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:52:50,353::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:52:50,359::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000297398 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:50,360::__init__::225::IOProcess::(_processLogs) (1647) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:50,361::__init__::225::IOProcess::(_processLogs) (1647) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:50,361::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1647) Building response >Thread-17::DEBUG::2014-08-22 09:52:50,362::__init__::225::IOProcess::(_processLogs) (1647) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:50,362::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:50,362::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:52:50,362::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:50,362::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:52:50,362::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) (1648) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) (1648) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) (1648) Building response >Thread-17::DEBUG::2014-08-22 09:52:50,363::__init__::225::IOProcess::(_processLogs) (1648) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:50,364::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:50,364::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:50,364::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:50,364::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:52:50,364::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:50,364::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) (1649) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) (1649) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) (1649) Building response >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) (1649) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:52:50,365::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:50,366::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:52:50,366::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:50,366::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:50,366::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) (1650) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) (1650) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) (1650) Building response >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) (1650) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:50,367::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:50,368::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:52:51,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:51,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122548 s, 83.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:53,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:53,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119976 s, 85.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:52:55,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:55,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115243 s, 88.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:56,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfec4217-9202-4848-9c4e-71a3d0ec616d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:56,984::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:52:56,984::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180022', 'lastCheck': '7.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000297398', 'lastCheck': '6.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:56,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`cfec4217-9202-4848-9c4e-71a3d0ec616d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000180022', 'lastCheck': '7.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000297398', 'lastCheck': '6.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:52:56,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfec4217-9202-4848-9c4e-71a3d0ec616d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:56,984::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:56,984::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:56,984::task::993::Storage.TaskManager.Task::(_decref) Task=`cfec4217-9202-4848-9c4e-71a3d0ec616d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:52:57,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:52:57,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117982 s, 86.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:52:57,217::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:52:57,217::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d21aaa9-e0fa-4c8c-9e95-060d12b30285`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:52:57,217::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:52:57,218::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:57,218::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d21aaa9-e0fa-4c8c-9e95-060d12b30285`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:52:57,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d21aaa9-e0fa-4c8c-9e95-060d12b30285`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:52:57,218::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:52:57,218::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:52:57,218::task::993::Storage.TaskManager.Task::(_decref) Task=`9d21aaa9-e0fa-4c8c-9e95-060d12b30285`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:52:57,293::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:52:57,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`444edb8b-ae0c-4177-a448-6dcd2a4f7931`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:52:57,294::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:52:57,294::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23c51f51-8529-41c3-ab78-5a2c1c5ba973`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:52:57,294::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:52:57,294::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:52:57,294::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23c51f51-8529-41c3-ab78-5a2c1c5ba973`::Granted request >Thread-46::DEBUG::2014-08-22 09:52:57,294::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`444edb8b-ae0c-4177-a448-6dcd2a4f7931`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:52:57,294::task::993::Storage.TaskManager.Task::(_decref) Task=`444edb8b-ae0c-4177-a448-6dcd2a4f7931`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:52:57,297::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:57,297::task::1191::Storage.TaskManager.Task::(prepare) Task=`444edb8b-ae0c-4177-a448-6dcd2a4f7931`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:52:57,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`444edb8b-ae0c-4177-a448-6dcd2a4f7931`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:52:57,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:52:57,297::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:52:57,297::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:52:57,297::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:52:57,297::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:52:57,297::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:52:57,297::task::993::Storage.TaskManager.Task::(_decref) Task=`444edb8b-ae0c-4177-a448-6dcd2a4f7931`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:59,201::__init__::225::IOProcess::(_processLogs) (1651) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:59,202::__init__::225::IOProcess::(_processLogs) (1651) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:59,202::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1651) Building response >Thread-21::DEBUG::2014-08-22 09:52:59,203::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:59,203::__init__::225::IOProcess::(_processLogs) (1651) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:59,210::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-45::DEBUG::2014-08-22 09:52:59,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 09:52:59,211::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:52:59,216::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189183 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) (1652) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:52:59,217::__init__::225::IOProcess::(_processLogs) (1652) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:52:59,225::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1652) Building response >Thread-17::DEBUG::2014-08-22 09:52:59,228::__init__::225::IOProcess::(_processLogs) (1652) Queuing response >Thread-17::DEBUG::2014-08-22 09:52:59,229::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:52:59,230::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:52:59,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163641 s, 62.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:00,367::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:00,367::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:00,368::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:00,368::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:00,369::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:53:00,369::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:00,369::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:00,375::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:53:00,376::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000217909 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:00,376::__init__::225::IOProcess::(_processLogs) (1653) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:00,377::__init__::225::IOProcess::(_processLogs) (1653) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:00,377::__init__::225::IOProcess::(_processLogs) (1653) Building response >Thread-17::DEBUG::2014-08-22 09:53:00,377::__init__::225::IOProcess::(_processLogs) (1653) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:00,378::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:00,379::__init__::225::IOProcess::(_processLogs) (1654) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:00,379::__init__::225::IOProcess::(_processLogs) (1654) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:00,379::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|(1654) Building response >Thread-17::DEBUG::2014-08-22 09:53:00,380::__init__::225::IOProcess::(_processLogs) (1654) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:00,380::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:00,380::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:00,380::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:00,380::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:53:00,380::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) (1655) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) (1655) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:00,381::__init__::225::IOProcess::(_processLogs) (1655) Building response >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) (1655) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:00,382::__init__::225::IOProcess::(_processLogs) (1656) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:00,383::__init__::225::IOProcess::(_processLogs) (1656) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:00,383::__init__::225::IOProcess::(_processLogs) (1656) Building response >Thread-17::DEBUG::2014-08-22 09:53:00,383::__init__::225::IOProcess::(_processLogs) (1656) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:00,383::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:53:00,383::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:00,384::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:00,384::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:53:00,384::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:00,384::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:00,384::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) (1657) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) (1657) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) (1657) Building response >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) (1657) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:00,385::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:53:01,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:01,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011727 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:03,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:03,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116557 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:05,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:05,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115672 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:07,393::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:53:07,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbb5c206-9c11-4bab-89ab-ae637f044c1e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:07,394::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:53:07,394::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:07,394::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbb5c206-9c11-4bab-89ab-ae637f044c1e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:07,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbb5c206-9c11-4bab-89ab-ae637f044c1e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:07,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:07,394::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:07,394::task::993::Storage.TaskManager.Task::(_decref) Task=`cbb5c206-9c11-4bab-89ab-ae637f044c1e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:07,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-46::DEBUG::2014-08-22 09:53:07,426::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:53:07,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`132ae20a-15ad-411a-9af1-6eda277b533d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:53:07,427::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:53:07,427::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`96058b64-1225-49c0-932b-e0fe007cb4a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:53:07,427::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:53:07,427::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:53:07,427::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`96058b64-1225-49c0-932b-e0fe007cb4a7`::Granted request >Thread-46::DEBUG::2014-08-22 09:53:07,428::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`132ae20a-15ad-411a-9af1-6eda277b533d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:53:07,428::task::993::Storage.TaskManager.Task::(_decref) Task=`132ae20a-15ad-411a-9af1-6eda277b533d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:53:07,430::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:07,430::task::1191::Storage.TaskManager.Task::(prepare) Task=`132ae20a-15ad-411a-9af1-6eda277b533d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:07,430::task::595::Storage.TaskManager.Task::(_updateState) Task=`132ae20a-15ad-411a-9af1-6eda277b533d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:53:07,430::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:53:07,430::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:53:07,430::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:53:07,431::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:53:07,431::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:53:07,431::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:53:07,432::task::993::Storage.TaskManager.Task::(_decref) Task=`132ae20a-15ad-411a-9af1-6eda277b533d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:07,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153468 s, 66.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:09,229::__init__::225::IOProcess::(_processLogs) (1652) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:09,229::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) (1658) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:09,230::__init__::225::IOProcess::(_processLogs) (1658) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:09,231::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1658) Building response >Thread-21::DEBUG::2014-08-22 09:53:09,231::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:09,232::__init__::225::IOProcess::(_processLogs) (1658) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:09,238::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:53:09,239::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000183674 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:09,239::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:09,240::__init__::225::IOProcess::(_processLogs) (1659) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:09,241::__init__::225::IOProcess::(_processLogs) (1659) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:09,241::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1659) Building response >Thread-17::DEBUG::2014-08-22 09:53:09,242::__init__::225::IOProcess::(_processLogs) (1659) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:09,242::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:09,242::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:53:09,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:09,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118884 s, 86.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:10,385::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:10,385::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) (1660) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) (1660) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:10,386::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1660) Building response >Thread-22::DEBUG::2014-08-22 09:53:10,387::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:10,387::__init__::225::IOProcess::(_processLogs) (1660) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:10,393::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:10,393::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:53:10,393::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000212856 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:10,394::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:10,394::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:10,394::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:10,394::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:10,394::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:10,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:10,395::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:10,395::__init__::225::IOProcess::(_processLogs) (1661) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:10,395::__init__::225::IOProcess::(_processLogs) (1661) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:10,395::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1661) Building response >Thread-17::DEBUG::2014-08-22 09:53:10,396::__init__::225::IOProcess::(_processLogs) (1661) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:10,396::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:10,396::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:10,396::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:10,396::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:53:10,397::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:10,397::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:10,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:10,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:10,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:10,397::__init__::225::IOProcess::(_processLogs) (1662) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) (1662) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) (1662) Building response >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) (1662) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:53:10,398::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:10,399::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:10,399::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:10,399::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:10,399::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:10,399::__init__::225::IOProcess::(_processLogs) (1663) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:10,399::__init__::225::IOProcess::(_processLogs) (1663) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) (1663) Building response >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) (1663) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:53:10,400::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:10,401::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:10,401::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:10,401::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:10,401::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:10,401::__init__::225::IOProcess::(_processLogs) (1664) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:10,401::__init__::225::IOProcess::(_processLogs) (1664) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:10,402::__init__::225::IOProcess::(_processLogs) (1664) Building response >Thread-17::DEBUG::2014-08-22 09:53:10,402::__init__::225::IOProcess::(_processLogs) (1664) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:10,402::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:10,402::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:53:11,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:11,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120852 s, 84.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:12,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac0e65b0-d9b4-41f2-9ca1-eed820acdece`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:12,222::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:53:12,222::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183674', 'lastCheck': '3.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000212856', 'lastCheck': '1.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:12,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac0e65b0-d9b4-41f2-9ca1-eed820acdece`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183674', 'lastCheck': '3.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000212856', 'lastCheck': '1.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:12,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac0e65b0-d9b4-41f2-9ca1-eed820acdece`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:12,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:12,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:12,223::task::993::Storage.TaskManager.Task::(_decref) Task=`ac0e65b0-d9b4-41f2-9ca1-eed820acdece`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:13,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:13,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117732 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:15,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:15,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114578 s, 89.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:17,479::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:53:17,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`96be0200-4d57-4660-a17e-cfcc5dc68c23`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:17,480::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:53:17,480::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:17,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`96be0200-4d57-4660-a17e-cfcc5dc68c23`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:17,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`96be0200-4d57-4660-a17e-cfcc5dc68c23`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:17,480::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:17,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:17,480::task::993::Storage.TaskManager.Task::(_decref) Task=`96be0200-4d57-4660-a17e-cfcc5dc68c23`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:53:17,513::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:53:17,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea66c0aa-127f-4357-be4d-a5c7f10b93d5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:53:17,514::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:53:17,514::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f5be74aa-c52f-4dcb-82fe-bc396ff884f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:53:17,514::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:53:17,514::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:53:17,514::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f5be74aa-c52f-4dcb-82fe-bc396ff884f7`::Granted request >Thread-46::DEBUG::2014-08-22 09:53:17,515::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea66c0aa-127f-4357-be4d-a5c7f10b93d5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:53:17,515::task::993::Storage.TaskManager.Task::(_decref) Task=`ea66c0aa-127f-4357-be4d-a5c7f10b93d5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:53:17,517::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:17,517::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea66c0aa-127f-4357-be4d-a5c7f10b93d5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:17,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea66c0aa-127f-4357-be4d-a5c7f10b93d5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:53:17,517::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:53:17,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:53:17,517::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:53:17,517::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:53:17,517::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:53:17,517::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:53:17,517::task::993::Storage.TaskManager.Task::(_decref) Task=`ea66c0aa-127f-4357-be4d-a5c7f10b93d5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:17,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:17,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117924 s, 86.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:19,242::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) (1665) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:19,243::__init__::225::IOProcess::(_processLogs) (1665) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:19,244::__init__::225::IOProcess::(_processLogs) (1665) Queuing response >Thread-21::DEBUG::2014-08-22 09:53:19,244::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:19,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:19,251::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:53:19,252::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000173654 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:19,252::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:19,252::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:19,252::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:19,253::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:19,253::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:19,253::__init__::225::IOProcess::(_processLogs) (1666) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:19,253::__init__::225::IOProcess::(_processLogs) (1666) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:19,253::__init__::225::IOProcess::(_processLogs) (1666) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:19,254::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:19,254::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:53:19,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:19,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118396 s, 86.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:20,402::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) (1667) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:20,403::__init__::225::IOProcess::(_processLogs) (1667) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:20,404::__init__::225::IOProcess::(_processLogs) (1667) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:20,404::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:53:20,404::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:20,405::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:53:20,411::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206906 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:20,411::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:20,412::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:20,412::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:20,412::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:20,412::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:20,412::__init__::225::IOProcess::(_processLogs) (1668) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:20,412::__init__::225::IOProcess::(_processLogs) (1668) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:20,413::__init__::225::IOProcess::(_processLogs) (1668) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:20,413::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:20,413::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) (1669) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:20,414::__init__::225::IOProcess::(_processLogs) (1669) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:20,415::__init__::225::IOProcess::(_processLogs) (1669) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:20,415::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:20,415::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:20,415::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:53:20,415::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) (1670) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) (1670) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) (1670) Building response >Thread-17::DEBUG::2014-08-22 09:53:20,416::__init__::225::IOProcess::(_processLogs) (1670) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:20,417::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:20,417::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:20,417::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:53:20,417::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:20,417::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) (1671) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) (1671) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) (1671) Building response >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) (1671) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:20,418::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:53:21,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:21,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121414 s, 84.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:23,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:23,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119613 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:25,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:25,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157997 s, 64.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:27,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`5022e3b4-afd5-4acb-89ce-78b204118da0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:27,454::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:53:27,454::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000173654', 'lastCheck': '8.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206906', 'lastCheck': '7.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:27,455::task::1191::Storage.TaskManager.Task::(prepare) Task=`5022e3b4-afd5-4acb-89ce-78b204118da0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000173654', 'lastCheck': '8.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206906', 'lastCheck': '7.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:27,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`5022e3b4-afd5-4acb-89ce-78b204118da0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:27,455::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:27,455::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:27,455::task::993::Storage.TaskManager.Task::(_decref) Task=`5022e3b4-afd5-4acb-89ce-78b204118da0`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:53:27,567::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:53:27,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1c42cb2-58ed-4249-9b79-cd1b5f458fd3`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:27,567::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:53:27,567::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:27,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1c42cb2-58ed-4249-9b79-cd1b5f458fd3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:27,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1c42cb2-58ed-4249-9b79-cd1b5f458fd3`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:27,568::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:27,568::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:27,568::task::993::Storage.TaskManager.Task::(_decref) Task=`c1c42cb2-58ed-4249-9b79-cd1b5f458fd3`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:53:27,613::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:53:27,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd1afc1e-dc15-4ba1-aed1-05c92335d716`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:53:27,613::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:53:27,613::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5219a2c2-6550-4eaa-bdca-47d94852b806`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:53:27,613::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:53:27,613::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:53:27,613::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5219a2c2-6550-4eaa-bdca-47d94852b806`::Granted request >Thread-46::DEBUG::2014-08-22 09:53:27,614::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bd1afc1e-dc15-4ba1-aed1-05c92335d716`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:53:27,614::task::993::Storage.TaskManager.Task::(_decref) Task=`bd1afc1e-dc15-4ba1-aed1-05c92335d716`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:53:27,616::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:27,616::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd1afc1e-dc15-4ba1-aed1-05c92335d716`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:27,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd1afc1e-dc15-4ba1-aed1-05c92335d716`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:53:27,616::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:53:27,616::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:53:27,616::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:53:27,616::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:53:27,616::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:53:27,616::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:53:27,617::task::993::Storage.TaskManager.Task::(_decref) Task=`bd1afc1e-dc15-4ba1-aed1-05c92335d716`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:27,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:27,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116966 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:29,255::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:29,255::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:29,255::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:29,255::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:29,255::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:29,256::__init__::225::IOProcess::(_processLogs) (1672) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:29,256::__init__::225::IOProcess::(_processLogs) (1672) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:29,256::__init__::225::IOProcess::(_processLogs) (1672) Queuing response >Thread-21::DEBUG::2014-08-22 09:53:29,257::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:29,257::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:29,263::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:53:29,264::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000235841 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) (1673) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:29,265::__init__::225::IOProcess::(_processLogs) (1673) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:29,266::__init__::225::IOProcess::(_processLogs) (1673) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:29,267::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:29,267::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:53:29,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:30,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116192 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:30,418::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) (1674) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) (1674) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:30,419::__init__::225::IOProcess::(_processLogs) (1674) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:30,420::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:53:30,420::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:30,421::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:53:30,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000195951 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:30,427::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:30,427::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:30,428::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:30,428::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:30,428::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:30,428::__init__::225::IOProcess::(_processLogs) (1675) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:30,428::__init__::225::IOProcess::(_processLogs) (1675) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:30,429::__init__::225::IOProcess::(_processLogs) (1675) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:30,429::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:30,429::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:30,430::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:53:30,430::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:30,430::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:30,430::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:30,430::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:30,431::__init__::225::IOProcess::(_processLogs) (1676) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:30,431::__init__::225::IOProcess::(_processLogs) (1676) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:30,431::__init__::225::IOProcess::(_processLogs) (1676) Building response >Thread-17::DEBUG::2014-08-22 09:53:30,431::__init__::225::IOProcess::(_processLogs) (1676) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:30,431::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:30,431::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:30,432::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:53:30,432::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:30,432::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:30,432::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) (1677) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) (1677) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) (1677) Building response >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) (1677) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:30,433::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:30,434::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:53:30,434::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:30,434::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) (1678) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) (1678) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) (1678) Building response >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) (1678) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:30,435::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:53:32,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:32,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219937 s, 46.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:34,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:34,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114262 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:36,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:36,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120449 s, 85.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:37,720::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:53:37,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c171e13-141a-4346-8f8e-db628435ee63`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:37,720::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:53:37,720::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:37,720::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c171e13-141a-4346-8f8e-db628435ee63`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:37,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c171e13-141a-4346-8f8e-db628435ee63`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:37,721::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:37,721::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:37,721::task::993::Storage.TaskManager.Task::(_decref) Task=`6c171e13-141a-4346-8f8e-db628435ee63`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:53:37,766::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:53:37,766::task::595::Storage.TaskManager.Task::(_updateState) Task=`0788758a-701a-417e-987e-3eacdf0c80d3`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:53:37,766::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:53:37,766::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d33a52cd-11f8-4b0b-bb57-6ff874884126`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:53:37,767::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:53:37,767::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:53:37,767::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d33a52cd-11f8-4b0b-bb57-6ff874884126`::Granted request >Thread-46::DEBUG::2014-08-22 09:53:37,767::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0788758a-701a-417e-987e-3eacdf0c80d3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:53:37,767::task::993::Storage.TaskManager.Task::(_decref) Task=`0788758a-701a-417e-987e-3eacdf0c80d3`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:53:37,769::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:37,769::task::1191::Storage.TaskManager.Task::(prepare) Task=`0788758a-701a-417e-987e-3eacdf0c80d3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:37,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`0788758a-701a-417e-987e-3eacdf0c80d3`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:53:37,769::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:53:37,769::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:53:37,769::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:53:37,770::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:53:37,770::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:53:37,770::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:53:37,770::task::993::Storage.TaskManager.Task::(_decref) Task=`0788758a-701a-417e-987e-3eacdf0c80d3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:38,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:38,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120585 s, 84.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:39,267::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:39,267::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:39,269::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:39,270::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:53:39,270::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:39,270::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:39,276::__init__::225::IOProcess::(_processLogs) (1679) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:39,276::__init__::225::IOProcess::(_processLogs) (1679) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:39,276::__init__::225::IOProcess::(_processLogs) (1679) Building response >Thread-17::DEBUG::2014-08-22 09:53:39,276::__init__::225::IOProcess::(_processLogs) (1679) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:39,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:39,277::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:53:39,279::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00116529 s, 341 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) (1680) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:39,280::__init__::225::IOProcess::(_processLogs) (1680) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:39,281::__init__::225::IOProcess::(_processLogs) (1680) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:39,281::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:53:40,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:40,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119937 s, 85.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:40,435::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) (1681) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) (1681) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:40,436::__init__::225::IOProcess::(_processLogs) (1681) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:40,437::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:53:40,437::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:40,438::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:53:40,444::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198221 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:40,444::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:40,444::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:40,445::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:40,445::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:40,445::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:40,445::__init__::225::IOProcess::(_processLogs) (1682) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:40,445::__init__::225::IOProcess::(_processLogs) (1682) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:40,446::__init__::225::IOProcess::(_processLogs) (1682) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:40,446::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:40,446::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:40,447::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:53:40,447::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:40,447::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:40,447::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:40,447::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) (1683) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) (1683) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) (1683) Building response >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) (1683) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:53:40,448::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) (1684) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) (1684) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) (1684) Building response >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) (1684) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:40,449::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:40,450::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:40,450::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:53:40,450::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:40,450::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) (1685) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) (1685) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) (1685) Building response >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) (1685) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:40,451::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:53:42,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:42,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011708 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:42,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`1addbc64-1b89-4a54-be7a-250fb5a16f68`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:42,790::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:53:42,790::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00116529', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198221', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:42,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`1addbc64-1b89-4a54-be7a-250fb5a16f68`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00116529', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198221', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:42,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`1addbc64-1b89-4a54-be7a-250fb5a16f68`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:42,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:42,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:42,790::task::993::Storage.TaskManager.Task::(_decref) Task=`1addbc64-1b89-4a54-be7a-250fb5a16f68`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:44,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:44,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115558 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:46,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:46,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119794 s, 85.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:47,815::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:53:47,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d3976e5-f79c-4b62-92f0-85d04e9c60ef`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:47,815::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:53:47,816::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:47,816::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d3976e5-f79c-4b62-92f0-85d04e9c60ef`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:47,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d3976e5-f79c-4b62-92f0-85d04e9c60ef`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:47,816::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:47,816::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:47,816::task::993::Storage.TaskManager.Task::(_decref) Task=`8d3976e5-f79c-4b62-92f0-85d04e9c60ef`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:53:47,869::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:53:47,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`9187d701-0bd4-4e1c-9c7e-abc71d370d21`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:53:47,870::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:53:47,870::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`51510d41-baa3-4608-bfb5-c468d4c51bd9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:53:47,870::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:53:47,870::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:53:47,870::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`51510d41-baa3-4608-bfb5-c468d4c51bd9`::Granted request >Thread-46::DEBUG::2014-08-22 09:53:47,870::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9187d701-0bd4-4e1c-9c7e-abc71d370d21`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:53:47,871::task::993::Storage.TaskManager.Task::(_decref) Task=`9187d701-0bd4-4e1c-9c7e-abc71d370d21`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:53:47,872::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:47,873::task::1191::Storage.TaskManager.Task::(prepare) Task=`9187d701-0bd4-4e1c-9c7e-abc71d370d21`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:47,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`9187d701-0bd4-4e1c-9c7e-abc71d370d21`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:53:47,873::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:53:47,873::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:53:47,873::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:53:47,873::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:53:47,873::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:53:47,873::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:53:47,873::task::993::Storage.TaskManager.Task::(_decref) Task=`9187d701-0bd4-4e1c-9c7e-abc71d370d21`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:48,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:48,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118459 s, 86.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:49,283::__init__::225::IOProcess::(_processLogs) (1686) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:49,284::__init__::225::IOProcess::(_processLogs) (1686) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:49,284::__init__::225::IOProcess::(_processLogs) (1686) Queuing response >Thread-21::DEBUG::2014-08-22 09:53:49,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:49,285::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:49,291::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:53:49,292::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000179231 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) (1687) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:49,293::__init__::225::IOProcess::(_processLogs) (1687) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:49,294::__init__::225::IOProcess::(_processLogs) (1687) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:49,294::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:49,294::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:50,451::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:50,451::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:50,452::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:50,452::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:50,452::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:50,452::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:50,452::__init__::225::IOProcess::(_processLogs) (1688) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:50,452::__init__::225::IOProcess::(_processLogs) (1688) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:50,453::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1688) Building response >Thread-17::DEBUG::2014-08-22 09:53:50,453::__init__::225::IOProcess::(_processLogs) (1688) Queuing response >Thread-22::DEBUG::2014-08-22 09:53:50,453::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:53:50,454::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:50,460::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:53:50,461::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00020216 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) (1689) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) (1689) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:50,462::__init__::225::IOProcess::(_processLogs) (1689) Building response >Thread-17::DEBUG::2014-08-22 09:53:50,463::__init__::225::IOProcess::(_processLogs) (1689) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:50,463::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:53:50,464::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:53:50,464::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:50,464::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:53:50,464::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) (1690) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) (1690) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) (1690) Building response >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) (1690) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:50,465::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:50,466::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:50,466::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:50,466::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:53:50,466::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:50,466::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) (1691) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) (1691) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) (1691) Building response >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) (1691) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:53:50,467::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:50,468::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:53:50,468::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:50,468::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:50,468::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) (1692) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) (1692) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) (1692) Building response >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) (1692) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:50,469::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:53:50,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:50,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117432 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:52,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:52,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119576 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:54,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:54,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116356 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:53:56,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:56,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118971 s, 86.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:53:57,919::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:53:57,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`71c5bba4-0d36-412e-bb8d-91b9fff109b0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:57,919::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:53:57,920::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:57,920::task::1191::Storage.TaskManager.Task::(prepare) Task=`71c5bba4-0d36-412e-bb8d-91b9fff109b0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:53:57,920::task::595::Storage.TaskManager.Task::(_updateState) Task=`71c5bba4-0d36-412e-bb8d-91b9fff109b0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:57,920::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:57,920::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:57,920::task::993::Storage.TaskManager.Task::(_decref) Task=`71c5bba4-0d36-412e-bb8d-91b9fff109b0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:53:57,953::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:53:57,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`cffa5cff-a7c4-4e8c-aa9e-aa82212dc737`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:53:57,953::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:53:57,953::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b4bff284-8690-4f70-bb4f-c1e12cfe7582`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:53:57,953::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:53:57,953::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:53:57,954::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b4bff284-8690-4f70-bb4f-c1e12cfe7582`::Granted request >Thread-46::DEBUG::2014-08-22 09:53:57,954::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cffa5cff-a7c4-4e8c-aa9e-aa82212dc737`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:53:57,954::task::993::Storage.TaskManager.Task::(_decref) Task=`cffa5cff-a7c4-4e8c-aa9e-aa82212dc737`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:53:57,956::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:57,956::task::1191::Storage.TaskManager.Task::(prepare) Task=`cffa5cff-a7c4-4e8c-aa9e-aa82212dc737`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:53:57,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`cffa5cff-a7c4-4e8c-aa9e-aa82212dc737`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:53:57,956::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:53:57,956::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:53:57,956::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:53:57,956::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:53:57,956::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:53:57,957::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:53:57,957::task::993::Storage.TaskManager.Task::(_decref) Task=`cffa5cff-a7c4-4e8c-aa9e-aa82212dc737`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:53:57,968::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6d3041c-456a-4c4b-8277-aab9823601f1`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:53:57,968::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:53:57,968::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000179231', 'lastCheck': '8.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020216', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:57,969::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6d3041c-456a-4c4b-8277-aab9823601f1`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000179231', 'lastCheck': '8.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00020216', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:53:57,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6d3041c-456a-4c4b-8277-aab9823601f1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:53:57,969::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:53:57,969::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:53:57,969::task::993::Storage.TaskManager.Task::(_decref) Task=`f6d3041c-456a-4c4b-8277-aab9823601f1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:53:58,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:53:58,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116303 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:59,295::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:59,295::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:59,295::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:59,295::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:59,295::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:59,295::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:59,296::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:59,296::__init__::225::IOProcess::(_processLogs) (1693) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:59,296::__init__::225::IOProcess::(_processLogs) (1693) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:59,296::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1693) Building response >Thread-17::DEBUG::2014-08-22 09:53:59,297::__init__::225::IOProcess::(_processLogs) (1693) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:59,297::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:59,297::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:53:59,297::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:53:59,303::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000248163 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:53:59,304::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:53:59,304::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:53:59,304::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:53:59,304::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:53:59,305::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:53:59,305::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:53:59,305::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:53:59,305::__init__::225::IOProcess::(_processLogs) (1694) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:53:59,305::__init__::225::IOProcess::(_processLogs) (1694) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:53:59,306::__init__::225::IOProcess::(_processLogs) (1694) Queuing response >Thread-17::DEBUG::2014-08-22 09:53:59,306::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:53:59,306::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:00,469::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:00,469::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:00,469::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:00,469::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:00,470::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:00,470::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:00,470::__init__::225::IOProcess::(_processLogs) (1695) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:00,470::__init__::225::IOProcess::(_processLogs) (1695) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:00,470::__init__::225::IOProcess::(_processLogs) DEBUG|(1695) Building response >Thread-17::DEBUG::2014-08-22 09:54:00,471::__init__::225::IOProcess::(_processLogs) (1695) Queuing response >Thread-22::DEBUG::2014-08-22 09:54:00,471::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:00,471::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:00,477::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:54:00,478::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021805 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:00,479::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:00,480::__init__::225::IOProcess::(_processLogs) (1696) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:00,480::__init__::225::IOProcess::(_processLogs) (1696) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:00,480::__init__::225::IOProcess::(_processLogs) DEBUG|(1696) Building response >Thread-17::DEBUG::2014-08-22 09:54:00,481::__init__::225::IOProcess::(_processLogs) (1696) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:00,481::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:00,481::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:00,481::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:00,481::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:54:00,482::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:00,482::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:00,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:00,482::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:00,482::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:00,482::__init__::225::IOProcess::(_processLogs) (1697) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) (1697) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) (1697) Building response >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) (1697) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:00,483::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) (1698) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) (1698) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) (1698) Building response >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) (1698) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:00,484::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:00,485::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:00,486::__init__::225::IOProcess::(_processLogs) (1699) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:00,486::__init__::225::IOProcess::(_processLogs) (1699) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:00,486::__init__::225::IOProcess::(_processLogs) DEBUG|(1699) Building response >Thread-17::DEBUG::2014-08-22 09:54:00,486::__init__::225::IOProcess::(_processLogs) (1699) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:00,487::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:00,487::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:54:00,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:00,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122133 s, 83.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:02,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:02,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209288 s, 48.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:04,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:05,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137834 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:07,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:07,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115313 s, 88.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:08,005::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:54:08,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc7a02d5-5eb8-4211-8b42-7dad8af420ab`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:08,007::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:54:08,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:08,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc7a02d5-5eb8-4211-8b42-7dad8af420ab`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:08,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc7a02d5-5eb8-4211-8b42-7dad8af420ab`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:08,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:08,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:08,007::task::993::Storage.TaskManager.Task::(_decref) Task=`dc7a02d5-5eb8-4211-8b42-7dad8af420ab`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:54:08,050::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:54:08,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`910e5f0c-ec18-4cc8-a6cb-cc4337700616`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:54:08,051::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:54:08,051::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ae817024-c0ed-40da-a1d5-795d621e5e8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:54:08,051::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:54:08,051::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:54:08,051::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ae817024-c0ed-40da-a1d5-795d621e5e8b`::Granted request >Thread-46::DEBUG::2014-08-22 09:54:08,052::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`910e5f0c-ec18-4cc8-a6cb-cc4337700616`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:54:08,053::task::993::Storage.TaskManager.Task::(_decref) Task=`910e5f0c-ec18-4cc8-a6cb-cc4337700616`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:54:08,054::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:08,055::task::1191::Storage.TaskManager.Task::(prepare) Task=`910e5f0c-ec18-4cc8-a6cb-cc4337700616`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:08,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`910e5f0c-ec18-4cc8-a6cb-cc4337700616`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:54:08,055::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:54:08,055::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:54:08,055::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:54:08,055::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:54:08,055::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:54:08,055::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:54:08,055::task::993::Storage.TaskManager.Task::(_decref) Task=`910e5f0c-ec18-4cc8-a6cb-cc4337700616`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:54:09,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:09,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115602 s, 88.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:09,306::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) (1700) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:09,307::__init__::225::IOProcess::(_processLogs) (1700) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:09,308::__init__::225::IOProcess::(_processLogs) DEBUG|(1700) Building response >Thread-21::DEBUG::2014-08-22 09:54:09,308::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:09,309::__init__::225::IOProcess::(_processLogs) (1700) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:09,315::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:54:09,316::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000189566 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:09,316::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:09,317::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:09,317::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:09,317::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:09,317::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:09,318::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:09,318::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:09,318::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:09,318::__init__::225::IOProcess::(_processLogs) (1701) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:09,318::__init__::225::IOProcess::(_processLogs) (1701) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:09,318::__init__::225::IOProcess::(_processLogs) DEBUG|(1701) Building response >Thread-17::DEBUG::2014-08-22 09:54:09,319::__init__::225::IOProcess::(_processLogs) (1701) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:09,319::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:09,319::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:10,487::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:10,487::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:10,487::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:10,487::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:10,487::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:10,488::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:10,488::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:10,488::__init__::225::IOProcess::(_processLogs) (1702) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:10,488::__init__::225::IOProcess::(_processLogs) (1702) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:10,488::__init__::225::IOProcess::(_processLogs) (1702) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:10,489::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:54:10,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:10,489::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:54:10,496::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000221857 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:10,496::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:10,496::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:10,496::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:10,496::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:10,497::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:10,497::__init__::225::IOProcess::(_processLogs) (1703) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:10,497::__init__::225::IOProcess::(_processLogs) (1703) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:10,497::__init__::225::IOProcess::(_processLogs) (1703) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:10,498::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:10,498::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:10,498::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:54:10,498::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) (1704) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) (1704) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) (1704) Building response >Thread-17::DEBUG::2014-08-22 09:54:10,499::__init__::225::IOProcess::(_processLogs) (1704) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:10,500::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:10,500::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:10,500::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:54:10,500::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:10,500::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) (1705) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) (1705) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) (1705) Building response >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) (1705) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:10,501::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:54:10,502::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:10,502::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:10,502::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:10,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:10,503::__init__::225::IOProcess::(_processLogs) (1706) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:10,503::__init__::225::IOProcess::(_processLogs) (1706) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:10,503::__init__::225::IOProcess::(_processLogs) (1706) Building response >Thread-17::DEBUG::2014-08-22 09:54:10,503::__init__::225::IOProcess::(_processLogs) (1706) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:10,503::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:10,503::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:54:11,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:11,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120852 s, 84.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:13,144::task::595::Storage.TaskManager.Task::(_updateState) Task=`f75f7876-5488-4571-a2be-12ddfc0fa0f2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:13,144::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:54:13,145::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189566', 'lastCheck': '3.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221857', 'lastCheck': '2.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:13,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`f75f7876-5488-4571-a2be-12ddfc0fa0f2`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000189566', 'lastCheck': '3.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221857', 'lastCheck': '2.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:13,146::task::595::Storage.TaskManager.Task::(_updateState) Task=`f75f7876-5488-4571-a2be-12ddfc0fa0f2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:13,146::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:13,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:13,146::task::993::Storage.TaskManager.Task::(_decref) Task=`f75f7876-5488-4571-a2be-12ddfc0fa0f2`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:54:13,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:13,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118217 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:15,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:15,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117579 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:17,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:17,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177633 s, 57.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:18,101::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:54:18,102::task::595::Storage.TaskManager.Task::(_updateState) Task=`3454305a-5fed-42ff-bd27-caf0cc59eded`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:18,102::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:54:18,102::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:18,102::task::1191::Storage.TaskManager.Task::(prepare) Task=`3454305a-5fed-42ff-bd27-caf0cc59eded`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:18,102::task::595::Storage.TaskManager.Task::(_updateState) Task=`3454305a-5fed-42ff-bd27-caf0cc59eded`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:18,102::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:18,102::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:18,102::task::993::Storage.TaskManager.Task::(_decref) Task=`3454305a-5fed-42ff-bd27-caf0cc59eded`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:54:18,126::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:54:18,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`830fe29f-359b-4eb0-90a9-c898bdd3edb1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:54:18,126::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:54:18,127::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c770f97f-b612-4412-a533-bbf4be90ef7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:54:18,127::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:54:18,127::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:54:18,127::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c770f97f-b612-4412-a533-bbf4be90ef7a`::Granted request >Thread-46::DEBUG::2014-08-22 09:54:18,127::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`830fe29f-359b-4eb0-90a9-c898bdd3edb1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:54:18,127::task::993::Storage.TaskManager.Task::(_decref) Task=`830fe29f-359b-4eb0-90a9-c898bdd3edb1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:54:18,129::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:18,129::task::1191::Storage.TaskManager.Task::(prepare) Task=`830fe29f-359b-4eb0-90a9-c898bdd3edb1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:18,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`830fe29f-359b-4eb0-90a9-c898bdd3edb1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:54:18,129::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:54:18,129::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:54:18,130::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:54:18,130::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:54:18,130::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:54:18,130::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:54:18,130::task::993::Storage.TaskManager.Task::(_decref) Task=`830fe29f-359b-4eb0-90a9-c898bdd3edb1`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) (1707) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:19,320::__init__::225::IOProcess::(_processLogs) (1707) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:19,321::__init__::225::IOProcess::(_processLogs) DEBUG|(1707) Building response >Thread-21::DEBUG::2014-08-22 09:54:19,322::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:19,322::__init__::225::IOProcess::(_processLogs) (1707) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:19,328::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:19,328::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:54:19,330::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000755885 s, 525 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:19,330::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:19,330::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:19,330::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:19,330::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:19,330::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:19,331::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:19,331::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:19,331::__init__::225::IOProcess::(_processLogs) (1708) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:19,331::__init__::225::IOProcess::(_processLogs) (1708) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:19,331::__init__::225::IOProcess::(_processLogs) DEBUG|(1708) Building response >Thread-17::DEBUG::2014-08-22 09:54:19,332::__init__::225::IOProcess::(_processLogs) (1708) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:19,332::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:19,333::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:54:19,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:19,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115853 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:20,503::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:20,504::__init__::225::IOProcess::(_processLogs) (1709) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:20,504::__init__::225::IOProcess::(_processLogs) (1709) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:20,504::__init__::225::IOProcess::(_processLogs) DEBUG|(1709) Building response >Thread-22::DEBUG::2014-08-22 09:54:20,505::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:20,505::__init__::225::IOProcess::(_processLogs) (1709) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:20,511::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:20,511::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:54:20,512::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223868 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:20,512::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:20,512::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) (1710) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) (1710) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) (1710) Building response >Thread-17::DEBUG::2014-08-22 09:54:20,513::__init__::225::IOProcess::(_processLogs) (1710) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:20,514::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:54:20,514::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:20,515::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:20,515::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:54:20,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:20,515::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) (1711) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) (1711) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) (1711) Building response >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) (1711) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:20,516::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:20,517::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:54:20,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:20,517::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:20,518::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:20,518::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:20,518::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:20,518::__init__::225::IOProcess::(_processLogs) (1712) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) (1712) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) (1712) Building response >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) (1712) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:54:20,519::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:20,520::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:20,520::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:20,520::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:20,520::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:20,520::__init__::225::IOProcess::(_processLogs) (1713) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:20,521::__init__::225::IOProcess::(_processLogs) (1713) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:20,521::__init__::225::IOProcess::(_processLogs) (1713) Building response >Thread-17::DEBUG::2014-08-22 09:54:20,521::__init__::225::IOProcess::(_processLogs) (1713) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:20,521::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:20,521::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:54:21,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:21,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119419 s, 85.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:23,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:23,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011588 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:25,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:25,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117261 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:27,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:27,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114356 s, 89.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:28,206::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:54:28,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`27fbad31-5083-448a-97e9-ee514c389bd1`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:28,206::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:54:28,206::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:28,206::task::1191::Storage.TaskManager.Task::(prepare) Task=`27fbad31-5083-448a-97e9-ee514c389bd1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:28,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`27fbad31-5083-448a-97e9-ee514c389bd1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:28,207::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:28,207::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:28,207::task::993::Storage.TaskManager.Task::(_decref) Task=`27fbad31-5083-448a-97e9-ee514c389bd1`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:54:28,247::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:54:28,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b090378-aaf9-4477-9070-3fb5d1b87d5d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:54:28,247::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:54:28,247::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6d9d64fe-7aae-4965-8af7-0915d113a55c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:54:28,247::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:54:28,248::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:54:28,248::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6d9d64fe-7aae-4965-8af7-0915d113a55c`::Granted request >Thread-46::DEBUG::2014-08-22 09:54:28,248::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9b090378-aaf9-4477-9070-3fb5d1b87d5d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:54:28,248::task::993::Storage.TaskManager.Task::(_decref) Task=`9b090378-aaf9-4477-9070-3fb5d1b87d5d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:54:28,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:28,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b090378-aaf9-4477-9070-3fb5d1b87d5d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:28,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b090378-aaf9-4477-9070-3fb5d1b87d5d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:54:28,250::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:54:28,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:54:28,251::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:54:28,251::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:54:28,251::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:54:28,251::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:54:28,251::task::993::Storage.TaskManager.Task::(_decref) Task=`9b090378-aaf9-4477-9070-3fb5d1b87d5d`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:54:28,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`467e7230-9cf7-4a3e-9b81-b5ab80d69b40`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:28,415::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:54:28,416::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000755885', 'lastCheck': '9.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223868', 'lastCheck': '7.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:28,416::task::1191::Storage.TaskManager.Task::(prepare) Task=`467e7230-9cf7-4a3e-9b81-b5ab80d69b40`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000755885', 'lastCheck': '9.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223868', 'lastCheck': '7.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:28,416::task::595::Storage.TaskManager.Task::(_updateState) Task=`467e7230-9cf7-4a3e-9b81-b5ab80d69b40`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:28,416::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:28,416::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:28,416::task::993::Storage.TaskManager.Task::(_decref) Task=`467e7230-9cf7-4a3e-9b81-b5ab80d69b40`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:29,333::__init__::225::IOProcess::(_processLogs) (1714) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:29,334::__init__::225::IOProcess::(_processLogs) (1714) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:29,334::__init__::225::IOProcess::(_processLogs) (1714) Building response >Thread-17::DEBUG::2014-08-22 09:54:29,334::__init__::225::IOProcess::(_processLogs) (1714) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:29,334::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:29,334::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:54:29,334::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:54:29,341::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000244865 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:29,341::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:29,341::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:29,341::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) (1715) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) (1715) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) (1715) Building response >Thread-17::DEBUG::2014-08-22 09:54:29,342::__init__::225::IOProcess::(_processLogs) (1715) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:29,343::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:54:29,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:29,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119387 s, 85.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:30,521::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:30,521::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:30,522::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:30,522::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:30,522::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:30,522::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:30,522::__init__::225::IOProcess::(_processLogs) (1716) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:30,522::__init__::225::IOProcess::(_processLogs) (1716) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:30,523::__init__::225::IOProcess::(_processLogs) DEBUG|(1716) Building response >Thread-17::DEBUG::2014-08-22 09:54:30,523::__init__::225::IOProcess::(_processLogs) (1716) Queuing response >Thread-22::DEBUG::2014-08-22 09:54:30,524::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:30,524::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:30,530::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:54:30,531::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000201721 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) (1717) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:30,532::__init__::225::IOProcess::(_processLogs) (1717) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:30,533::__init__::225::IOProcess::(_processLogs) DEBUG|(1717) Building response >Thread-17::DEBUG::2014-08-22 09:54:30,534::__init__::225::IOProcess::(_processLogs) (1717) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:30,534::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:30,534::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:30,534::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:30,534::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:54:30,534::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) (1718) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) (1718) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:30,535::__init__::225::IOProcess::(_processLogs) (1718) Building response >Thread-17::DEBUG::2014-08-22 09:54:30,536::__init__::225::IOProcess::(_processLogs) (1718) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:30,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:30,536::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:30,536::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:30,536::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:54:30,536::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) (1719) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) (1719) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) (1719) Building response >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) (1719) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:30,537::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:30,538::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:30,538::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:54:30,538::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:30,538::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) (1720) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) (1720) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) (1720) Building response >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) (1720) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:30,539::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:54:31,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:31,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011884 s, 86.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:33,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:33,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121411 s, 84.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:35,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:35,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115492 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:37,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:37,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115791 s, 88.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:38,307::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:54:38,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`754ec524-3b7d-4e38-a2d5-3297579c8bb3`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:38,307::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:54:38,308::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:38,308::task::1191::Storage.TaskManager.Task::(prepare) Task=`754ec524-3b7d-4e38-a2d5-3297579c8bb3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:38,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`754ec524-3b7d-4e38-a2d5-3297579c8bb3`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:38,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:38,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:38,308::task::993::Storage.TaskManager.Task::(_decref) Task=`754ec524-3b7d-4e38-a2d5-3297579c8bb3`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:54:38,360::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:54:38,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4688fa6-551c-4a6e-904e-b8c9611f7dc2`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:54:38,360::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:54:38,360::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`768dda46-fe3f-420e-b4b5-90c9f0eef08c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:54:38,360::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:54:38,360::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:54:38,361::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`768dda46-fe3f-420e-b4b5-90c9f0eef08c`::Granted request >Thread-46::DEBUG::2014-08-22 09:54:38,361::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d4688fa6-551c-4a6e-904e-b8c9611f7dc2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:54:38,361::task::993::Storage.TaskManager.Task::(_decref) Task=`d4688fa6-551c-4a6e-904e-b8c9611f7dc2`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:54:38,363::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:38,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4688fa6-551c-4a6e-904e-b8c9611f7dc2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:38,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4688fa6-551c-4a6e-904e-b8c9611f7dc2`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:54:38,363::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:54:38,363::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:54:38,363::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:54:38,363::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:54:38,364::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:54:38,364::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:54:38,364::task::993::Storage.TaskManager.Task::(_decref) Task=`d4688fa6-551c-4a6e-904e-b8c9611f7dc2`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:54:39,344::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:54:39,344::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:39,344::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:39,344::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:39,344::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:39,344::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:39,345::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:39,345::__init__::225::IOProcess::(_processLogs) (1721) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:39,345::__init__::225::IOProcess::(_processLogs) (1721) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:39,345::__init__::225::IOProcess::(_processLogs) (1721) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:39,346::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:54:39,346::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:39,347::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:54:39,353::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000210683 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) (1722) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:39,354::__init__::225::IOProcess::(_processLogs) (1722) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:39,355::__init__::225::IOProcess::(_processLogs) (1722) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:39,356::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:39,356::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:54:39,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:39,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119303 s, 85.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:40,539::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:40,539::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:40,540::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:40,540::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:40,540::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:40,540::__init__::225::IOProcess::(_processLogs) (1723) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:40,540::__init__::225::IOProcess::(_processLogs) (1723) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:40,540::__init__::225::IOProcess::(_processLogs) (1723) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:40,541::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:54:40,541::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:40,541::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:54:40,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000192258 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) (1724) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:40,548::__init__::225::IOProcess::(_processLogs) (1724) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:40,549::__init__::225::IOProcess::(_processLogs) (1724) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:40,550::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:40,550::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:40,550::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:54:40,550::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:40,550::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) (1725) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) (1725) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) (1725) Building response >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) (1725) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:40,551::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:40,552::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:54:40,552::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:40,552::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:40,552::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:40,552::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:40,552::__init__::225::IOProcess::(_processLogs) (1726) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) (1726) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) (1726) Building response >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) (1726) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:54:40,553::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:40,554::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:40,554::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:40,554::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:40,554::__init__::225::IOProcess::(_processLogs) (1727) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:40,554::__init__::225::IOProcess::(_processLogs) (1727) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:40,555::__init__::225::IOProcess::(_processLogs) (1727) Building response >Thread-17::DEBUG::2014-08-22 09:54:40,555::__init__::225::IOProcess::(_processLogs) (1727) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:40,555::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:40,555::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:54:41,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:41,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118502 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:43,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`287fc55e-41d2-4d0c-8b55-7b8bb50acb2c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:43,612::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:54:43,612::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210683', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000192258', 'lastCheck': '3.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:43,612::task::1191::Storage.TaskManager.Task::(prepare) Task=`287fc55e-41d2-4d0c-8b55-7b8bb50acb2c`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210683', 'lastCheck': '4.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000192258', 'lastCheck': '3.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:43,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`287fc55e-41d2-4d0c-8b55-7b8bb50acb2c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:43,612::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:43,612::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:43,612::task::993::Storage.TaskManager.Task::(_decref) Task=`287fc55e-41d2-4d0c-8b55-7b8bb50acb2c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:54:43,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:43,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114606 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:45,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:46,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118334 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:48,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:48,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0127852 s, 80.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:48,412::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:54:48,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`144092b2-ba9e-4bfc-a2e4-f3bb4d57ef93`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:48,412::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:54:48,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:48,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`144092b2-ba9e-4bfc-a2e4-f3bb4d57ef93`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:48,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`144092b2-ba9e-4bfc-a2e4-f3bb4d57ef93`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:48,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:48,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:48,413::task::993::Storage.TaskManager.Task::(_decref) Task=`144092b2-ba9e-4bfc-a2e4-f3bb4d57ef93`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:54:48,454::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:54:48,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`491ccd20-7e11-4799-9b7d-fc5fe7a6717e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:54:48,454::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:54:48,455::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c80f8f35-bd57-42a7-89f8-08bba945dfe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:54:48,455::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:54:48,455::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:54:48,455::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c80f8f35-bd57-42a7-89f8-08bba945dfe0`::Granted request >Thread-46::DEBUG::2014-08-22 09:54:48,455::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`491ccd20-7e11-4799-9b7d-fc5fe7a6717e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:54:48,455::task::993::Storage.TaskManager.Task::(_decref) Task=`491ccd20-7e11-4799-9b7d-fc5fe7a6717e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:54:48,457::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:48,457::task::1191::Storage.TaskManager.Task::(prepare) Task=`491ccd20-7e11-4799-9b7d-fc5fe7a6717e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:48,457::task::595::Storage.TaskManager.Task::(_updateState) Task=`491ccd20-7e11-4799-9b7d-fc5fe7a6717e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:54:48,457::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:54:48,458::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:54:48,458::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:54:48,458::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:54:48,458::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:54:48,458::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:54:48,458::task::993::Storage.TaskManager.Task::(_decref) Task=`491ccd20-7e11-4799-9b7d-fc5fe7a6717e`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:49,356::__init__::225::IOProcess::(_processLogs) (1728) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:49,357::__init__::225::IOProcess::(_processLogs) (1728) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:49,357::__init__::225::IOProcess::(_processLogs) (1728) Queuing response >Thread-21::DEBUG::2014-08-22 09:54:49,358::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:49,358::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:49,364::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:54:49,365::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000251671 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:49,366::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:49,366::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:49,366::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:49,366::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:49,366::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:49,366::__init__::225::IOProcess::(_processLogs) (1729) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:49,367::__init__::225::IOProcess::(_processLogs) (1729) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:49,367::__init__::225::IOProcess::(_processLogs) (1729) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:49,368::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:49,368::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:54:50,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:50,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116098 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) (1730) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:50,555::__init__::225::IOProcess::(_processLogs) (1730) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:50,556::__init__::225::IOProcess::(_processLogs) (1730) Queuing response >Thread-22::DEBUG::2014-08-22 09:54:50,557::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:50,557::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:50,563::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:54:50,564::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000202297 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) (1731) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:50,565::__init__::225::IOProcess::(_processLogs) (1731) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:50,566::__init__::225::IOProcess::(_processLogs) (1731) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:50,566::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:50,567::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:54:50,567::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:54:50,567::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:50,567::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) (1732) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) (1732) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) (1732) Building response >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) (1732) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:54:50,568::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:50,569::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:50,569::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:50,569::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:50,569::__init__::225::IOProcess::(_processLogs) (1733) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:50,569::__init__::225::IOProcess::(_processLogs) (1733) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:50,569::__init__::225::IOProcess::(_processLogs) (1733) Building response >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) (1733) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:50,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) (1734) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) (1734) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) (1734) Building response >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) (1734) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:50,571::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:54:52,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:52,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120265 s, 85.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:54,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:54,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116932 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:56,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:56,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113014 s, 90.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:54:58,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:54:58,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113744 s, 90.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:54:58,582::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:54:58,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`db917a41-a838-4564-a84f-786deab2e5ad`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:58,583::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:54:58,583::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:58,583::task::1191::Storage.TaskManager.Task::(prepare) Task=`db917a41-a838-4564-a84f-786deab2e5ad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:54:58,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`db917a41-a838-4564-a84f-786deab2e5ad`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:58,583::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:58,583::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:58,583::task::993::Storage.TaskManager.Task::(_decref) Task=`db917a41-a838-4564-a84f-786deab2e5ad`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:54:58,616::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:54:58,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`848a3525-65bd-4749-ba9a-2c69869a07fc`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:54:58,616::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:54:58,617::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e80922df-ad12-4919-8d44-726a3d949674`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:54:58,617::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:54:58,617::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:54:58,617::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e80922df-ad12-4919-8d44-726a3d949674`::Granted request >Thread-46::DEBUG::2014-08-22 09:54:58,617::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`848a3525-65bd-4749-ba9a-2c69869a07fc`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:54:58,617::task::993::Storage.TaskManager.Task::(_decref) Task=`848a3525-65bd-4749-ba9a-2c69869a07fc`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:54:58,619::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:58,619::task::1191::Storage.TaskManager.Task::(prepare) Task=`848a3525-65bd-4749-ba9a-2c69869a07fc`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:54:58,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`848a3525-65bd-4749-ba9a-2c69869a07fc`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:54:58,619::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:54:58,620::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:54:58,620::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:54:58,620::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:54:58,620::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:54:58,620::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:54:58,620::task::993::Storage.TaskManager.Task::(_decref) Task=`848a3525-65bd-4749-ba9a-2c69869a07fc`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:54:58,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`f899e7af-3941-4ed9-80df-2d76039cf151`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:54:58,796::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:54:58,796::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000251671', 'lastCheck': '9.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000202297', 'lastCheck': '8.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:58,796::task::1191::Storage.TaskManager.Task::(prepare) Task=`f899e7af-3941-4ed9-80df-2d76039cf151`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000251671', 'lastCheck': '9.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000202297', 'lastCheck': '8.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:54:58,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`f899e7af-3941-4ed9-80df-2d76039cf151`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:54:58,796::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:54:58,796::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:54:58,797::task::993::Storage.TaskManager.Task::(_decref) Task=`f899e7af-3941-4ed9-80df-2d76039cf151`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:54:59,368::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:59,368::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:54:59,369::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:59,369::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:59,369::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:59,369::__init__::225::IOProcess::(_processLogs) (1735) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:59,369::__init__::225::IOProcess::(_processLogs) (1735) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:59,369::__init__::225::IOProcess::(_processLogs) (1735) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:59,370::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:54:59,370::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:54:59,371::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:54:59,377::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000170857 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) (1736) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:54:59,378::__init__::225::IOProcess::(_processLogs) (1736) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:54:59,379::__init__::225::IOProcess::(_processLogs) (1736) Queuing response >Thread-17::DEBUG::2014-08-22 09:54:59,380::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:54:59,380::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:55:00,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:00,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118065 s, 86.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:00,571::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) (1737) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) (1737) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:00,572::__init__::225::IOProcess::(_processLogs) DEBUG|(1737) Building response >Thread-22::DEBUG::2014-08-22 09:55:00,573::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:00,573::__init__::225::IOProcess::(_processLogs) (1737) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:00,579::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:55:00,580::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000196219 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:00,581::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:00,582::__init__::225::IOProcess::(_processLogs) (1738) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:00,583::__init__::225::IOProcess::(_processLogs) (1738) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:00,583::__init__::225::IOProcess::(_processLogs) DEBUG|(1738) Building response >Thread-17::DEBUG::2014-08-22 09:55:00,584::__init__::225::IOProcess::(_processLogs) (1738) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:00,584::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:00,584::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:00,584::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:00,584::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:55:00,584::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) (1739) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) (1739) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) (1739) Building response >Thread-17::DEBUG::2014-08-22 09:55:00,585::__init__::225::IOProcess::(_processLogs) (1739) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:00,586::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:00,586::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:00,586::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:00,586::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:55:00,586::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:00,586::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) (1740) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) (1740) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) (1740) Building response >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) (1740) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:00,587::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:00,588::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:00,588::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:55:00,588::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:00,588::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:00,588::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) (1741) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) (1741) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) (1741) Building response >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) (1741) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:00,589::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:55:02,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:02,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208546 s, 49.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:04,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:04,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116126 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:06,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:06,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191111 s, 53.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:08,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:08,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132023 s, 77.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:08,688::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:55:08,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`f20c9d2f-cf73-4410-b39d-e5f7eb9f0e78`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:08,690::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:55:08,690::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:08,691::task::1191::Storage.TaskManager.Task::(prepare) Task=`f20c9d2f-cf73-4410-b39d-e5f7eb9f0e78`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:08,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`f20c9d2f-cf73-4410-b39d-e5f7eb9f0e78`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:08,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:08,692::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:08,692::task::993::Storage.TaskManager.Task::(_decref) Task=`f20c9d2f-cf73-4410-b39d-e5f7eb9f0e78`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:55:08,735::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:55:08,736::task::595::Storage.TaskManager.Task::(_updateState) Task=`074d0893-c87c-4ec9-837d-2670e5133bf7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:55:08,736::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:55:08,736::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`74fa8de6-e90c-4bc4-a529-128020d947ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:55:08,736::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:55:08,736::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:55:08,736::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`74fa8de6-e90c-4bc4-a529-128020d947ce`::Granted request >Thread-46::DEBUG::2014-08-22 09:55:08,737::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`074d0893-c87c-4ec9-837d-2670e5133bf7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:55:08,737::task::993::Storage.TaskManager.Task::(_decref) Task=`074d0893-c87c-4ec9-837d-2670e5133bf7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:55:08,739::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:08,739::task::1191::Storage.TaskManager.Task::(prepare) Task=`074d0893-c87c-4ec9-837d-2670e5133bf7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:08,739::task::595::Storage.TaskManager.Task::(_updateState) Task=`074d0893-c87c-4ec9-837d-2670e5133bf7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:55:08,739::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:55:08,739::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:55:08,739::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:55:08,739::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:55:08,739::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:55:08,739::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:55:08,740::task::993::Storage.TaskManager.Task::(_decref) Task=`074d0893-c87c-4ec9-837d-2670e5133bf7`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:55:09,380::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:09,380::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:09,380::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) (1742) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) (1742) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:09,381::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1742) Building response >Thread-17::DEBUG::2014-08-22 09:55:09,382::__init__::225::IOProcess::(_processLogs) (1742) Queuing response >Thread-21::DEBUG::2014-08-22 09:55:09,382::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:09,383::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:09,389::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:55:09,390::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000197232 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) (1743) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:09,391::__init__::225::IOProcess::(_processLogs) (1743) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:09,392::__init__::225::IOProcess::(_processLogs) (1743) Building response >Thread-17::DEBUG::2014-08-22 09:55:09,392::__init__::225::IOProcess::(_processLogs) (1743) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:09,392::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:09,392::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:10,589::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:10,589::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) (1744) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:10,590::__init__::225::IOProcess::(_processLogs) (1744) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:10,591::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1744) Building response >Thread-22::DEBUG::2014-08-22 09:55:10,592::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:10,592::__init__::225::IOProcess::(_processLogs) (1744) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:10,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:55:10,599::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000254618 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:10,599::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:10,600::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:10,600::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:10,600::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:10,600::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:10,600::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:10,601::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:10,601::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:10,601::__init__::225::IOProcess::(_processLogs) (1745) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:10,601::__init__::225::IOProcess::(_processLogs) (1745) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:10,602::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1745) Building response >Thread-17::DEBUG::2014-08-22 09:55:10,602::__init__::225::IOProcess::(_processLogs) (1745) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:10,602::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:10,602::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:10,603::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:10,603::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) (1746) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:10,604::__init__::225::IOProcess::(_processLogs) (1746) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) (1746) Building response >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) (1746) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:55:10,605::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) (1747) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) (1747) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) (1747) Building response >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) (1747) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:10,606::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:10,607::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:10,607::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:55:10,607::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:10,607::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:10,607::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:10,607::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:10,608::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:10,608::__init__::225::IOProcess::(_processLogs) (1748) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:10,608::__init__::225::IOProcess::(_processLogs) (1748) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:10,608::__init__::225::IOProcess::(_processLogs) (1748) Building response >Thread-17::DEBUG::2014-08-22 09:55:10,608::__init__::225::IOProcess::(_processLogs) (1748) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:10,609::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:10,609::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:55:10,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:10,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117188 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:12,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:12,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115146 s, 88.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:13,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc97f002-3696-4c80-ac81-e742625ff615`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:13,987::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:55:13,987::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000197232', 'lastCheck': '4.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000254618', 'lastCheck': '3.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:13,987::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc97f002-3696-4c80-ac81-e742625ff615`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000197232', 'lastCheck': '4.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000254618', 'lastCheck': '3.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:13,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc97f002-3696-4c80-ac81-e742625ff615`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:13,988::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:13,988::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:13,988::task::993::Storage.TaskManager.Task::(_decref) Task=`fc97f002-3696-4c80-ac81-e742625ff615`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:55:14,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:14,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115897 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:16,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:16,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118878 s, 86.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:18,801::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:55:18,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`19708266-6acc-48b1-be2e-0432add344c4`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:18,801::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:55:18,801::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:18,802::task::1191::Storage.TaskManager.Task::(prepare) Task=`19708266-6acc-48b1-be2e-0432add344c4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:18,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`19708266-6acc-48b1-be2e-0432add344c4`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:18,802::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:18,802::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:18,802::task::993::Storage.TaskManager.Task::(_decref) Task=`19708266-6acc-48b1-be2e-0432add344c4`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:55:18,871::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:55:18,871::task::595::Storage.TaskManager.Task::(_updateState) Task=`98a3a2e4-d62a-442f-92bf-4e8dd6bd561f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:55:18,872::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:55:18,872::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d41be987-e3e4-4955-b5f5-eb69b88ee449`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:55:18,872::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:55:18,872::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:55:18,872::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d41be987-e3e4-4955-b5f5-eb69b88ee449`::Granted request >Thread-46::DEBUG::2014-08-22 09:55:18,872::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`98a3a2e4-d62a-442f-92bf-4e8dd6bd561f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:55:18,872::task::993::Storage.TaskManager.Task::(_decref) Task=`98a3a2e4-d62a-442f-92bf-4e8dd6bd561f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:55:18,874::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:18,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`98a3a2e4-d62a-442f-92bf-4e8dd6bd561f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:18,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`98a3a2e4-d62a-442f-92bf-4e8dd6bd561f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:55:18,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:55:18,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:55:18,875::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:55:18,875::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:55:18,875::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:55:18,875::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:55:18,875::task::993::Storage.TaskManager.Task::(_decref) Task=`98a3a2e4-d62a-442f-92bf-4e8dd6bd561f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:55:18,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:18,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117683 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:19,393::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:19,393::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:19,393::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:19,393::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:19,393::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:19,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:19,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:19,394::__init__::225::IOProcess::(_processLogs) (1749) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:19,394::__init__::225::IOProcess::(_processLogs) (1749) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:19,394::__init__::225::IOProcess::(_processLogs) (1749) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:19,395::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:55:19,395::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:19,396::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:55:19,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000152447 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) (1750) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:19,403::__init__::225::IOProcess::(_processLogs) (1750) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:19,404::__init__::225::IOProcess::(_processLogs) (1750) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:19,405::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:19,405::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:20,609::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:20,609::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:20,609::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:20,609::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:20,610::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:20,610::__init__::225::IOProcess::(_processLogs) (1751) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:20,610::__init__::225::IOProcess::(_processLogs) (1751) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:20,612::__init__::225::IOProcess::(_processLogs) (1751) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:20,612::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:55:20,613::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:55:20,619::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000250811 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) (1752) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:20,620::__init__::225::IOProcess::(_processLogs) (1752) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:20,621::__init__::225::IOProcess::(_processLogs) (1752) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) (1753) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:20,622::__init__::225::IOProcess::(_processLogs) (1753) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:20,623::__init__::225::IOProcess::(_processLogs) (1753) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:20,623::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:20,623::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:20,623::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:55:20,623::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:20,624::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:20,624::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:20,624::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:20,624::__init__::225::IOProcess::(_processLogs) (1754) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:20,624::__init__::225::IOProcess::(_processLogs) (1754) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:20,624::__init__::225::IOProcess::(_processLogs) (1754) Building response >Thread-17::DEBUG::2014-08-22 09:55:20,625::__init__::225::IOProcess::(_processLogs) (1754) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:20,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:20,625::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:20,625::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:55:20,625::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:20,625::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) (1755) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) (1755) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) (1755) Building response >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) (1755) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:20,626::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:55:20,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:20,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115964 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:23,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:23,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114006 s, 89.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:25,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:25,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011534 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:27,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:27,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011546 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:28,960::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:55:28,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a16ef7b-a579-4ffb-a7db-218103e4de49`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:28,961::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:55:28,961::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:28,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a16ef7b-a579-4ffb-a7db-218103e4de49`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:28,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a16ef7b-a579-4ffb-a7db-218103e4de49`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:28,961::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:28,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:28,962::task::993::Storage.TaskManager.Task::(_decref) Task=`7a16ef7b-a579-4ffb-a7db-218103e4de49`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:55:28,984::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:55:28,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8662884-161f-41dc-862f-4da7bad66ef6`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:55:28,985::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:55:28,985::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7fffa1be-b348-4084-9852-5191218a76ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:55:28,985::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:55:28,985::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:55:28,985::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7fffa1be-b348-4084-9852-5191218a76ca`::Granted request >Thread-46::DEBUG::2014-08-22 09:55:28,985::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b8662884-161f-41dc-862f-4da7bad66ef6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:55:28,985::task::993::Storage.TaskManager.Task::(_decref) Task=`b8662884-161f-41dc-862f-4da7bad66ef6`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:55:28,987::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:28,988::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8662884-161f-41dc-862f-4da7bad66ef6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:28,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8662884-161f-41dc-862f-4da7bad66ef6`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:55:28,988::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:55:28,988::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:55:28,988::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:55:28,988::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:55:28,988::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:55:28,988::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:55:28,988::task::993::Storage.TaskManager.Task::(_decref) Task=`b8662884-161f-41dc-862f-4da7bad66ef6`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:55:29,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:29,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112762 s, 90.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:29,194::task::595::Storage.TaskManager.Task::(_updateState) Task=`3725fa79-0204-4898-a0f7-71648c49e1f7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:29,195::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:55:29,196::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000152447', 'lastCheck': '9.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000250811', 'lastCheck': '8.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:29,196::task::1191::Storage.TaskManager.Task::(prepare) Task=`3725fa79-0204-4898-a0f7-71648c49e1f7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000152447', 'lastCheck': '9.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000250811', 'lastCheck': '8.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:29,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`3725fa79-0204-4898-a0f7-71648c49e1f7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:29,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:29,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:29,198::task::993::Storage.TaskManager.Task::(_decref) Task=`3725fa79-0204-4898-a0f7-71648c49e1f7`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:55:29,405::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:29,405::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:29,405::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) (1756) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) (1756) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) (1756) Building response >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) (1756) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:29,406::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:55:29,407::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:55:29,413::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000222164 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) (1757) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:29,414::__init__::225::IOProcess::(_processLogs) (1757) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:29,415::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1757) Building response >Thread-17::DEBUG::2014-08-22 09:55:29,416::__init__::225::IOProcess::(_processLogs) (1757) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:29,416::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:29,416::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:30,627::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:30,627::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:30,627::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:30,627::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:30,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:55:30,628::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:30,629::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:30,635::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:55:30,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000194441 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:30,636::__init__::225::IOProcess::(_processLogs) (1758) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) (1758) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) (1758) Building response >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) (1758) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:30,637::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) (1759) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:30,638::__init__::225::IOProcess::(_processLogs) (1759) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:30,639::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1759) Building response >Thread-17::DEBUG::2014-08-22 09:55:30,639::__init__::225::IOProcess::(_processLogs) (1759) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:30,639::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:30,639::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:30,640::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:30,640::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:55:30,640::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:30,640::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) (1760) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) (1760) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) (1760) Building response >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) (1760) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:30,641::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:55:30,642::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:30,642::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:30,642::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:30,642::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:30,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:30,642::__init__::225::IOProcess::(_processLogs) (1761) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) (1761) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) (1761) Building response >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) (1761) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:55:30,643::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:30,644::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:30,644::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:30,644::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:30,644::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:30,645::__init__::225::IOProcess::(_processLogs) (1762) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:30,645::__init__::225::IOProcess::(_processLogs) (1762) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:30,645::__init__::225::IOProcess::(_processLogs) (1762) Building response >Thread-17::DEBUG::2014-08-22 09:55:30,645::__init__::225::IOProcess::(_processLogs) (1762) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:30,645::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:30,645::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:55:31,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:31,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133963 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:33,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:33,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114623 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:35,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:35,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011347 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:37,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:37,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011491 s, 89.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:39,069::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:55:39,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7d8dbbd-e871-447b-b53e-f4f3dcf6bf41`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:39,070::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:55:39,070::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:39,070::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7d8dbbd-e871-447b-b53e-f4f3dcf6bf41`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:39,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7d8dbbd-e871-447b-b53e-f4f3dcf6bf41`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:39,070::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:39,070::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:39,070::task::993::Storage.TaskManager.Task::(_decref) Task=`e7d8dbbd-e871-447b-b53e-f4f3dcf6bf41`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:55:39,107::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:55:39,107::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a188d26-fa4a-4b18-a367-76564fc6a73f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:55:39,107::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:55:39,108::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`652f1622-d48c-4fe3-9a62-e24597f46916`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:55:39,108::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:55:39,108::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:55:39,108::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`652f1622-d48c-4fe3-9a62-e24597f46916`::Granted request >Thread-46::DEBUG::2014-08-22 09:55:39,108::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5a188d26-fa4a-4b18-a367-76564fc6a73f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:55:39,108::task::993::Storage.TaskManager.Task::(_decref) Task=`5a188d26-fa4a-4b18-a367-76564fc6a73f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:55:39,110::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:39,110::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a188d26-fa4a-4b18-a367-76564fc6a73f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:39,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a188d26-fa4a-4b18-a367-76564fc6a73f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:55:39,111::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:55:39,111::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:55:39,111::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:55:39,111::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:55:39,111::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:55:39,111::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:55:39,111::task::993::Storage.TaskManager.Task::(_decref) Task=`5a188d26-fa4a-4b18-a367-76564fc6a73f`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:55:39,416::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:39,416::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:39,416::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:39,416::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:39,417::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:39,417::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:39,417::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:39,417::__init__::225::IOProcess::(_processLogs) (1763) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:39,417::__init__::225::IOProcess::(_processLogs) (1763) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:39,417::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1763) Building response >Thread-17::DEBUG::2014-08-22 09:55:39,418::__init__::225::IOProcess::(_processLogs) (1763) Queuing response >Thread-21::DEBUG::2014-08-22 09:55:39,418::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:39,419::__init__::225::IOProcess::(_processLogs) Generating json... >Dummy-45::DEBUG::2014-08-22 09:55:39,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:39,426::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:55:39,432::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000213912 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) (1764) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:39,433::__init__::225::IOProcess::(_processLogs) (1764) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:39,441::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1764) Building response >Thread-17::DEBUG::2014-08-22 09:55:39,443::__init__::225::IOProcess::(_processLogs) (1764) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:39,444::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:39,445::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:55:39,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154459 s, 66.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:40,645::__init__::225::IOProcess::(_processLogs) (1764) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:40,645::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:40,645::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:40,645::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:40,645::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:40,645::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:40,646::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:40,646::__init__::225::IOProcess::(_processLogs) (1765) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:40,646::__init__::225::IOProcess::(_processLogs) (1765) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:40,646::__init__::225::IOProcess::(_processLogs) (1765) Queuing response >Thread-22::DEBUG::2014-08-22 09:55:40,647::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:40,647::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:40,653::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:55:40,654::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000381364 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) (1766) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:40,655::__init__::225::IOProcess::(_processLogs) (1766) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:40,656::__init__::225::IOProcess::(_processLogs) (1766) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:40,657::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:40,657::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:40,657::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:55:40,657::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:40,657::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) (1767) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) (1767) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) (1767) Building response >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) (1767) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:40,658::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) (1768) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) (1768) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:40,659::__init__::225::IOProcess::(_processLogs) (1768) Building response >Thread-17::DEBUG::2014-08-22 09:55:40,660::__init__::225::IOProcess::(_processLogs) (1768) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:40,660::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:40,660::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:40,660::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:55:40,660::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:40,660::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) (1769) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) (1769) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) (1769) Building response >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) (1769) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:40,661::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:40,662::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:55:41,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:41,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116539 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:43,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:43,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113972 s, 89.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:44,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd2ac91e-5b38-4f4e-b05a-20ebc5569b75`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:44,402::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:55:44,403::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000213912', 'lastCheck': '5.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000381364', 'lastCheck': '3.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:44,403::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd2ac91e-5b38-4f4e-b05a-20ebc5569b75`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000213912', 'lastCheck': '5.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000381364', 'lastCheck': '3.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:44,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd2ac91e-5b38-4f4e-b05a-20ebc5569b75`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:44,403::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:44,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:44,403::task::993::Storage.TaskManager.Task::(_decref) Task=`dd2ac91e-5b38-4f4e-b05a-20ebc5569b75`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:55:45,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:45,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112988 s, 90.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:47,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:47,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0142276 s, 72.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:49,163::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:55:49,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`56eb9e57-2c20-48fb-a2aa-c6e6a30d9b15`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:49,163::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:55:49,163::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:49,164::task::1191::Storage.TaskManager.Task::(prepare) Task=`56eb9e57-2c20-48fb-a2aa-c6e6a30d9b15`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:49,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`56eb9e57-2c20-48fb-a2aa-c6e6a30d9b15`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:49,164::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:49,164::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:49,164::task::993::Storage.TaskManager.Task::(_decref) Task=`56eb9e57-2c20-48fb-a2aa-c6e6a30d9b15`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:55:49,203::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:55:49,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`325a64d3-29f7-402e-a7fe-c3d3611b87d8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:55:49,203::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:55:49,204::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3d415837-6364-4e13-a789-e96f3087880d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:55:49,204::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:55:49,204::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:55:49,204::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3d415837-6364-4e13-a789-e96f3087880d`::Granted request >Thread-46::DEBUG::2014-08-22 09:55:49,204::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`325a64d3-29f7-402e-a7fe-c3d3611b87d8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:55:49,204::task::993::Storage.TaskManager.Task::(_decref) Task=`325a64d3-29f7-402e-a7fe-c3d3611b87d8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:55:49,206::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:49,206::task::1191::Storage.TaskManager.Task::(prepare) Task=`325a64d3-29f7-402e-a7fe-c3d3611b87d8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:49,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`325a64d3-29f7-402e-a7fe-c3d3611b87d8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:55:49,206::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:55:49,206::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:55:49,207::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:55:49,207::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:55:49,207::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:55:49,207::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:55:49,207::task::993::Storage.TaskManager.Task::(_decref) Task=`325a64d3-29f7-402e-a7fe-c3d3611b87d8`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) (1770) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:49,445::__init__::225::IOProcess::(_processLogs) (1770) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:49,446::__init__::225::IOProcess::(_processLogs) (1770) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:49,447::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:55:49,447::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:49,447::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:55:49,454::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000146585 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:49,454::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:49,454::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:49,454::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:49,455::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:49,455::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:49,455::__init__::225::IOProcess::(_processLogs) (1771) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:49,455::__init__::225::IOProcess::(_processLogs) (1771) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:49,456::__init__::225::IOProcess::(_processLogs) (1771) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:49,456::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:49,456::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:55:49,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:49,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120882 s, 84.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:50,662::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:50,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:50,663::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:50,663::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:50,663::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:50,663::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:50,663::__init__::225::IOProcess::(_processLogs) (1772) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:50,663::__init__::225::IOProcess::(_processLogs) (1772) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:50,664::__init__::225::IOProcess::(_processLogs) DEBUG|(1772) Building response >Thread-22::DEBUG::2014-08-22 09:55:50,665::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:55:50,665::__init__::225::IOProcess::(_processLogs) (1772) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:50,671::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:55:50,672::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206439 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:50,672::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:50,673::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:50,673::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:55:50,673::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:50,674::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:50,674::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:50,674::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:50,674::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:50,674::__init__::225::IOProcess::(_processLogs) (1773) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:50,674::__init__::225::IOProcess::(_processLogs) (1773) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:50,675::__init__::225::IOProcess::(_processLogs) DEBUG|(1773) Building response >Thread-17::DEBUG::2014-08-22 09:55:50,675::__init__::225::IOProcess::(_processLogs) (1773) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:50,675::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:50,675::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:55:50,676::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:50,676::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:55:50,676::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:50,676::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:50,676::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:50,676::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) (1774) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) (1774) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) (1774) Building response >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) (1774) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:55:50,677::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:50,678::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:50,678::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:50,678::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:50,678::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:50,678::__init__::225::IOProcess::(_processLogs) (1775) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:50,678::__init__::225::IOProcess::(_processLogs) (1775) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) (1775) Building response >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) (1775) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:55:50,679::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) (1776) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) (1776) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:55:50,680::__init__::225::IOProcess::(_processLogs) (1776) Building response >Thread-17::DEBUG::2014-08-22 09:55:50,681::__init__::225::IOProcess::(_processLogs) (1776) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:50,681::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:50,681::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:55:51,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:51,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121198 s, 84.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:53,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:53,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205334 s, 49.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:55,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:55,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114188 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:55:57,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:57,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01132 s, 90.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:55:59,276::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-47::DEBUG::2014-08-22 09:55:59,277::task::595::Storage.TaskManager.Task::(_updateState) Task=`547a1469-320e-416c-8d2c-33b0c2ab784c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:59,277::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:55:59,277::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:59,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`547a1469-320e-416c-8d2c-33b0c2ab784c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:55:59,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`547a1469-320e-416c-8d2c-33b0c2ab784c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:59,278::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:59,278::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:59,278::task::993::Storage.TaskManager.Task::(_decref) Task=`547a1469-320e-416c-8d2c-33b0c2ab784c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:55:59,320::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-46::DEBUG::2014-08-22 09:55:59,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`51fa3cf0-dd4a-42c1-9845-41a26c72a619`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:55:59,320::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:55:59,320::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3903cebe-1791-499f-9645-82ba1242b951`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:55:59,321::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:55:59,321::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:55:59,321::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3903cebe-1791-499f-9645-82ba1242b951`::Granted request >Thread-46::DEBUG::2014-08-22 09:55:59,321::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`51fa3cf0-dd4a-42c1-9845-41a26c72a619`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:55:59,321::task::993::Storage.TaskManager.Task::(_decref) Task=`51fa3cf0-dd4a-42c1-9845-41a26c72a619`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:55:59,329::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:59,329::task::1191::Storage.TaskManager.Task::(prepare) Task=`51fa3cf0-dd4a-42c1-9845-41a26c72a619`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:55:59,329::task::595::Storage.TaskManager.Task::(_updateState) Task=`51fa3cf0-dd4a-42c1-9845-41a26c72a619`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:55:59,329::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:55:59,329::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:55:59,330::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:55:59,330::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:55:59,330::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:55:59,330::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:55:59,330::task::993::Storage.TaskManager.Task::(_decref) Task=`51fa3cf0-dd4a-42c1-9845-41a26c72a619`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:55:59,458::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:59,458::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:59,458::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:59,458::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:59,458::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:59,458::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:59,459::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:59,459::__init__::225::IOProcess::(_processLogs) (1777) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:59,459::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1777) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:59,460::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1777) Building response >Thread-17::DEBUG::2014-08-22 09:55:59,460::__init__::225::IOProcess::(_processLogs) (1777) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:59,461::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:55:59,462::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:55:59,462::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:55:59,470::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000366847 s, 1.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) (1778) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:55:59,471::__init__::225::IOProcess::(_processLogs) (1778) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:55:59,472::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1778) Building response >Thread-17::DEBUG::2014-08-22 09:55:59,473::__init__::225::IOProcess::(_processLogs) (1778) Queuing response >Thread-17::DEBUG::2014-08-22 09:55:59,473::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:55:59,473::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:55:59,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`f724fe0b-5356-45fb-b806-646e04864659`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:55:59,598::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:55:59,598::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000366847', 'lastCheck': '0.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206439', 'lastCheck': '8.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:59,598::task::1191::Storage.TaskManager.Task::(prepare) Task=`f724fe0b-5356-45fb-b806-646e04864659`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000366847', 'lastCheck': '0.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206439', 'lastCheck': '8.9', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:55:59,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`f724fe0b-5356-45fb-b806-646e04864659`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:55:59,598::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:55:59,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:55:59,599::task::993::Storage.TaskManager.Task::(_decref) Task=`f724fe0b-5356-45fb-b806-646e04864659`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:55:59,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:55:59,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115465 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:00,684::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:00,684::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:00,684::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:00,685::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:56:00,686::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:00,686::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) (1779) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) (1779) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) (1779) Building response >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) (1779) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:00,695::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:56:00,699::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00430831 s, 132 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:00,700::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) (1780) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:00,701::__init__::225::IOProcess::(_processLogs) (1780) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:00,702::__init__::225::IOProcess::(_processLogs) DEBUG|(1780) Building response >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) (1780) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:00,703::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:00,704::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:00,704::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:00,704::__init__::225::IOProcess::(_processLogs) (1781) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:00,704::__init__::225::IOProcess::(_processLogs) (1781) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:00,704::__init__::225::IOProcess::(_processLogs) (1781) Building response >Thread-17::DEBUG::2014-08-22 09:56:00,704::__init__::225::IOProcess::(_processLogs) (1781) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:00,705::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:56:00,706::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:00,706::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:00,706::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:56:00,706::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:00,706::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:00,706::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) (1782) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) (1782) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) (1782) Building response >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) (1782) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:00,707::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:56:00,708::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:00,708::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:00,708::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:56:00,708::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:00,708::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:00,709::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:00,709::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:00,709::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:00,709::__init__::225::IOProcess::(_processLogs) (1783) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:00,709::__init__::225::IOProcess::(_processLogs) (1783) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:00,709::__init__::225::IOProcess::(_processLogs) (1783) Building response >Thread-17::DEBUG::2014-08-22 09:56:00,710::__init__::225::IOProcess::(_processLogs) (1783) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:00,710::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:00,710::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:56:01,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:02,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214223 s, 47.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:04,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:04,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021236 s, 48.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:06,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:06,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115855 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:08,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:08,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011609 s, 88.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:56:09,439::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:56:09,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ca26991-f7f1-4985-af0b-382d54d0c9a1`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:09,439::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:56:09,439::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:09,439::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ca26991-f7f1-4985-af0b-382d54d0c9a1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:09,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ca26991-f7f1-4985-af0b-382d54d0c9a1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:09,440::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:09,440::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:09,440::task::993::Storage.TaskManager.Task::(_decref) Task=`3ca26991-f7f1-4985-af0b-382d54d0c9a1`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:56:09,473::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:09,473::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) (1784) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:09,474::__init__::225::IOProcess::(_processLogs) (1784) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:09,475::__init__::225::IOProcess::(_processLogs) DEBUG|(1784) Building response >Thread-21::DEBUG::2014-08-22 09:56:09,475::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:09,476::__init__::225::IOProcess::(_processLogs) (1784) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:09,482::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:09,483::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:56:09,483::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00022121 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:09,484::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:09,484::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:09,484::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:09,485::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:09,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-46::DEBUG::2014-08-22 09:56:09,486::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-17::DEBUG::2014-08-22 09:56:09,486::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-46::DEBUG::2014-08-22 09:56:09,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`570edfb4-99e2-40ec-bf2b-bc1f004fadd4`::moving from state init -> state preparing >Thread-17::DEBUG::2014-08-22 09:56:09,487::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-46::INFO::2014-08-22 09:56:09,487::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:56:09,487::__init__::225::IOProcess::(_processLogs) (1785) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:09,488::__init__::225::IOProcess::(_processLogs) (1785) Got request for method 'statvfs' >Thread-46::DEBUG::2014-08-22 09:56:09,489::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`872d207e-76be-496b-8e97-1ad917c81bd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:56:09,489::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-17::DEBUG::2014-08-22 09:56:09,490::__init__::225::IOProcess::(_processLogs) DEBUG|(1785) Building response >Thread-46::DEBUG::2014-08-22 09:56:09,491::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-17::DEBUG::2014-08-22 09:56:09,492::__init__::225::IOProcess::(_processLogs) (1785) Queuing response >Thread-46::DEBUG::2014-08-22 09:56:09,492::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`872d207e-76be-496b-8e97-1ad917c81bd4`::Granted request >Thread-17::DEBUG::2014-08-22 09:56:09,492::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:09,493::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-46::DEBUG::2014-08-22 09:56:09,493::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`570edfb4-99e2-40ec-bf2b-bc1f004fadd4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:56:09,493::task::993::Storage.TaskManager.Task::(_decref) Task=`570edfb4-99e2-40ec-bf2b-bc1f004fadd4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:56:09,495::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:09,496::task::1191::Storage.TaskManager.Task::(prepare) Task=`570edfb4-99e2-40ec-bf2b-bc1f004fadd4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:09,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`570edfb4-99e2-40ec-bf2b-bc1f004fadd4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:56:09,496::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:56:09,496::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:56:09,496::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:56:09,496::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:56:09,496::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:56:09,496::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:56:09,496::task::993::Storage.TaskManager.Task::(_decref) Task=`570edfb4-99e2-40ec-bf2b-bc1f004fadd4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:56:10,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:10,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115684 s, 88.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:10,716::__init__::225::IOProcess::(_processLogs) (1786) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:10,717::__init__::225::IOProcess::(_processLogs) (1786) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:10,717::__init__::225::IOProcess::(_processLogs) (1786) Queuing response >Thread-22::DEBUG::2014-08-22 09:56:10,718::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:10,718::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:10,724::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:56:10,725::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000254432 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) (1787) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:10,726::__init__::225::IOProcess::(_processLogs) (1787) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:10,727::__init__::225::IOProcess::(_processLogs) (1787) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:10,728::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:10,728::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:10,728::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:56:10,728::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:10,728::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) (1788) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) (1788) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) (1788) Building response >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) (1788) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:10,729::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:56:10,730::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:10,730::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:10,730::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:10,730::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:10,730::__init__::225::IOProcess::(_processLogs) (1789) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:10,730::__init__::225::IOProcess::(_processLogs) (1789) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) (1789) Building response >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) (1789) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:10,731::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) (1790) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) (1790) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) (1790) Building response >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) (1790) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:10,732::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:10,733::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:56:12,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:12,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116296 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:14,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:14,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115768 s, 88.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:56:14,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2611b22-f178-4d9f-8b71-4c1a1d2333a7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:14,819::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:56:14,820::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00022121', 'lastCheck': '5.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000254432', 'lastCheck': '4.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:56:14,820::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2611b22-f178-4d9f-8b71-4c1a1d2333a7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00022121', 'lastCheck': '5.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000254432', 'lastCheck': '4.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:56:14,820::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2611b22-f178-4d9f-8b71-4c1a1d2333a7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:14,820::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:14,820::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:14,821::task::993::Storage.TaskManager.Task::(_decref) Task=`b2611b22-f178-4d9f-8b71-4c1a1d2333a7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:56:16,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:16,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113723 s, 90.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:18,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:18,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115942 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:19,492::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:19,492::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:19,492::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:19,492::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:19,493::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:19,493::__init__::225::IOProcess::(_processLogs) (1791) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:19,493::__init__::225::IOProcess::(_processLogs) (1791) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:19,493::__init__::225::IOProcess::(_processLogs) (1791) Queuing response >Thread-21::DEBUG::2014-08-22 09:56:19,494::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:19,494::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:19,501::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:56:19,501::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000156715 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:19,502::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:19,502::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:19,502::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:19,502::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:19,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:19,503::__init__::225::IOProcess::(_processLogs) (1792) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:19,503::__init__::225::IOProcess::(_processLogs) (1792) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:19,503::__init__::225::IOProcess::(_processLogs) (1792) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:19,504::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:19,504::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:56:19,556::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:56:19,556::task::595::Storage.TaskManager.Task::(_updateState) Task=`dda01ffc-c8b4-4495-a5c6-a156c8979c5b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:19,557::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:56:19,557::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:19,557::task::1191::Storage.TaskManager.Task::(prepare) Task=`dda01ffc-c8b4-4495-a5c6-a156c8979c5b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:19,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`dda01ffc-c8b4-4495-a5c6-a156c8979c5b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:19,557::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:19,557::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:19,557::task::993::Storage.TaskManager.Task::(_decref) Task=`dda01ffc-c8b4-4495-a5c6-a156c8979c5b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:56:19,581::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:56:19,581::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6b949f4-669d-4c02-ab78-1b6b0db9c8f7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:56:19,581::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:56:19,581::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`39c7b3b6-038a-4d24-84ea-689957e2e6d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:56:19,582::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:56:19,582::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:56:19,582::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`39c7b3b6-038a-4d24-84ea-689957e2e6d6`::Granted request >Thread-46::DEBUG::2014-08-22 09:56:19,582::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6b949f4-669d-4c02-ab78-1b6b0db9c8f7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:56:19,582::task::993::Storage.TaskManager.Task::(_decref) Task=`b6b949f4-669d-4c02-ab78-1b6b0db9c8f7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:56:19,584::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:19,584::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6b949f4-669d-4c02-ab78-1b6b0db9c8f7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:19,584::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6b949f4-669d-4c02-ab78-1b6b0db9c8f7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:56:19,584::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:56:19,584::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:56:19,584::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:56:19,585::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:56:19,585::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:56:19,585::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:56:19,585::task::993::Storage.TaskManager.Task::(_decref) Task=`b6b949f4-669d-4c02-ab78-1b6b0db9c8f7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:56:20,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:20,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114619 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) (1793) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) (1793) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:20,733::__init__::225::IOProcess::(_processLogs) (1793) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:20,734::__init__::225::IOProcess::(_processLogs) (1793) Queuing responseDEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:56:20,734::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:56:20,735::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:56:20,741::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000276297 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:20,742::__init__::225::IOProcess::(_processLogs) (1793) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:20,742::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:20,742::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:20,742::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:20,742::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:20,742::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:20,743::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:20,743::__init__::225::IOProcess::(_processLogs) (1794) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:20,743::__init__::225::IOProcess::(_processLogs) (1794) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:20,743::__init__::225::IOProcess::(_processLogs) (1794) Building response >Thread-17::DEBUG::2014-08-22 09:56:20,744::__init__::225::IOProcess::(_processLogs) (1794) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:20,744::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:20,744::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:20,744::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:56:20,744::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:20,745::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:20,745::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:20,745::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:20,745::__init__::225::IOProcess::(_processLogs) (1795) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:20,745::__init__::225::IOProcess::(_processLogs) (1795) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:20,745::__init__::225::IOProcess::(_processLogs) (1795) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:20,746::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:20,746::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:20,746::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:56:20,746::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:20,746::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) (1796) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) (1796) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) (1796) Building response >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) (1796) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:20,747::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:20,748::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:56:20,748::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:20,748::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:20,748::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) (1797) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) (1797) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) (1797) Building response >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) (1797) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:20,749::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:56:22,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:22,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117471 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:24,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:24,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115966 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:26,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:26,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114829 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:28,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:28,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0110812 s, 92.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) (1798) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:29,505::__init__::225::IOProcess::(_processLogs) (1798) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:29,506::__init__::225::IOProcess::(_processLogs) (1798) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:29,507::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:56:29,507::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:29,508::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:56:29,514::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000192428 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:29,514::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) (1799) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:29,515::__init__::225::IOProcess::(_processLogs) (1799) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:29,516::__init__::225::IOProcess::(_processLogs) (1799) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:29,516::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:29,517::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:56:29,633::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:56:29,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc8dcbff-8c87-435f-b5ab-ba3a377d4678`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:29,634::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:56:29,634::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:29,634::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc8dcbff-8c87-435f-b5ab-ba3a377d4678`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:29,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc8dcbff-8c87-435f-b5ab-ba3a377d4678`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:29,634::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:29,634::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:29,634::task::993::Storage.TaskManager.Task::(_decref) Task=`fc8dcbff-8c87-435f-b5ab-ba3a377d4678`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:56:29,674::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:56:29,674::task::595::Storage.TaskManager.Task::(_updateState) Task=`45acf756-7fc8-4244-9b04-d00312b6885c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:56:29,674::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:56:29,675::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`946a90ec-23e6-463d-9625-6be3d8d7f0da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:56:29,675::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:56:29,675::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:56:29,675::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`946a90ec-23e6-463d-9625-6be3d8d7f0da`::Granted request >Thread-46::DEBUG::2014-08-22 09:56:29,675::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`45acf756-7fc8-4244-9b04-d00312b6885c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:56:29,675::task::993::Storage.TaskManager.Task::(_decref) Task=`45acf756-7fc8-4244-9b04-d00312b6885c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:56:29,677::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:29,677::task::1191::Storage.TaskManager.Task::(prepare) Task=`45acf756-7fc8-4244-9b04-d00312b6885c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:29,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`45acf756-7fc8-4244-9b04-d00312b6885c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:56:29,677::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:56:29,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:56:29,678::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:56:29,678::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:56:29,678::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:56:29,678::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:56:29,678::task::993::Storage.TaskManager.Task::(_decref) Task=`45acf756-7fc8-4244-9b04-d00312b6885c`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:56:30,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`1596b49c-3beb-4817-b302-ab7c264f3443`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:30,008::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:56:30,008::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000192428', 'lastCheck': '0.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000276297', 'lastCheck': '9.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:56:30,008::task::1191::Storage.TaskManager.Task::(prepare) Task=`1596b49c-3beb-4817-b302-ab7c264f3443`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000192428', 'lastCheck': '0.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000276297', 'lastCheck': '9.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:56:30,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`1596b49c-3beb-4817-b302-ab7c264f3443`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:30,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:30,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:30,008::task::993::Storage.TaskManager.Task::(_decref) Task=`1596b49c-3beb-4817-b302-ab7c264f3443`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:56:30,749::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:30,749::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:30,749::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:30,749::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:30,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:30,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:30,750::__init__::225::IOProcess::(_processLogs) (1800) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:30,750::__init__::225::IOProcess::(_processLogs) (1800) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:30,750::__init__::225::IOProcess::(_processLogs) (1800) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:30,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:56:30,751::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:30,751::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:56:30,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000236647 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:30,758::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:30,758::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:30,758::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:30,758::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:30,758::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:30,759::__init__::225::IOProcess::(_processLogs) (1801) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:30,759::__init__::225::IOProcess::(_processLogs) (1801) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:30,759::__init__::225::IOProcess::(_processLogs) (1801) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:30,760::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:30,760::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:30,760::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:56:30,760::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) (1802) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) (1802) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) (1802) Building response >Thread-17::DEBUG::2014-08-22 09:56:30,761::__init__::225::IOProcess::(_processLogs) (1802) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:30,762::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:30,762::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:30,762::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:56:30,762::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:30,762::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) (1803) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) (1803) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) (1803) Building response >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) (1803) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:30,763::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:56:30,764::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:30,764::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:30,764::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) (1804) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) (1804) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) (1804) Building response >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) (1804) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:30,765::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:56:30,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:30,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115923 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:32,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:32,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204364 s, 50.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:34,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:35,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113243 s, 90.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:37,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:37,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115639 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:39,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:39,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114666 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:39,517::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:39,517::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:39,517::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:39,517::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:39,517::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:39,517::__init__::225::IOProcess::(_processLogs) (1805) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:39,518::__init__::225::IOProcess::(_processLogs) (1805) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:39,518::__init__::225::IOProcess::(_processLogs) (1805) Queuing response >Thread-21::DEBUG::2014-08-22 09:56:39,519::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:39,519::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:39,525::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:56:39,526::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000183262 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) (1806) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:39,527::__init__::225::IOProcess::(_processLogs) (1806) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:39,528::__init__::225::IOProcess::(_processLogs) (1806) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:39,529::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:39,529::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:56:39,730::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:56:39,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1606009-8115-4f20-804e-9388164ac6a6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:39,730::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:56:39,730::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:39,730::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1606009-8115-4f20-804e-9388164ac6a6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:39,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1606009-8115-4f20-804e-9388164ac6a6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:39,731::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:39,731::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:39,731::task::993::Storage.TaskManager.Task::(_decref) Task=`e1606009-8115-4f20-804e-9388164ac6a6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:56:39,775::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:56:39,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`d07c71c1-b67f-41ee-8e7d-04142e2b9554`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:56:39,775::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:56:39,776::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c41314a4-04ba-452f-ba16-373435cac2d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:56:39,776::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:56:39,776::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:56:39,776::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c41314a4-04ba-452f-ba16-373435cac2d4`::Granted request >Thread-46::DEBUG::2014-08-22 09:56:39,776::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d07c71c1-b67f-41ee-8e7d-04142e2b9554`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:56:39,776::task::993::Storage.TaskManager.Task::(_decref) Task=`d07c71c1-b67f-41ee-8e7d-04142e2b9554`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:56:39,778::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:39,778::task::1191::Storage.TaskManager.Task::(prepare) Task=`d07c71c1-b67f-41ee-8e7d-04142e2b9554`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:39,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`d07c71c1-b67f-41ee-8e7d-04142e2b9554`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:56:39,778::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:56:39,779::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:56:39,779::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:56:39,779::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:56:39,779::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:56:39,779::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:56:39,779::task::993::Storage.TaskManager.Task::(_decref) Task=`d07c71c1-b67f-41ee-8e7d-04142e2b9554`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) (1807) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:40,765::__init__::225::IOProcess::(_processLogs) (1807) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:40,766::__init__::225::IOProcess::(_processLogs) (1807) Building response >Thread-17::DEBUG::2014-08-22 09:56:40,766::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:40,766::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:56:40,767::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:56:40,773::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0001563 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:40,774::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:40,774::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:40,774::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:40,774::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:40,775::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:40,775::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:40,775::__init__::225::IOProcess::(_processLogs) (1808) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:40,775::__init__::225::IOProcess::(_processLogs) (1808) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:40,775::__init__::225::IOProcess::(_processLogs) (1808) Building response >Thread-17::DEBUG::2014-08-22 09:56:40,776::__init__::225::IOProcess::(_processLogs) (1808) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:40,776::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:40,776::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:40,776::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:40,776::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:56:40,776::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) (1809) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) (1809) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) (1809) Building response >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) (1809) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:40,777::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:40,778::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:40,778::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:56:40,778::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:40,778::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:40,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) (1810) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) (1810) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) (1810) Building response >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) (1810) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:40,779::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:56:40,780::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:40,780::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:40,780::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) (1811) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) (1811) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) (1811) Building response >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) (1811) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:40,781::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:56:41,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:41,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115437 s, 88.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:43,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:43,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118238 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:45,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-47::DEBUG::2014-08-22 09:56:45,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b2e03ce-909f-4816-ac9d-13c63be58158`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:45,239::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:56:45,241::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183262', 'lastCheck': '5.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0001563', 'lastCheck': '4.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:56:45,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b2e03ce-909f-4816-ac9d-13c63be58158`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183262', 'lastCheck': '5.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0001563', 'lastCheck': '4.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:56:45,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b2e03ce-909f-4816-ac9d-13c63be58158`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:45,243::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:45,243::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:45,244::task::993::Storage.TaskManager.Task::(_decref) Task=`2b2e03ce-909f-4816-ac9d-13c63be58158`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:56:45,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208627 s, 49.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:47,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:47,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115724 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:49,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:49,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113039 s, 90.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:49,529::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:49,529::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:49,529::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) (1812) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) (1812) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:49,530::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1812) Building response >Thread-17::DEBUG::2014-08-22 09:56:49,531::__init__::225::IOProcess::(_processLogs) (1812) Queuing response >Thread-21::DEBUG::2014-08-22 09:56:49,531::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:49,532::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:49,538::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:56:49,539::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.0001928 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:49,540::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:49,541::__init__::225::IOProcess::(_processLogs) (1813) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:49,541::__init__::225::IOProcess::(_processLogs) (1813) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:49,541::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1813) Building response >Thread-17::DEBUG::2014-08-22 09:56:49,542::__init__::225::IOProcess::(_processLogs) (1813) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:49,542::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:49,542::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:56:49,834::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:56:49,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`297526c6-826f-4028-a450-6e9279a17547`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:49,834::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:56:49,834::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:49,834::task::1191::Storage.TaskManager.Task::(prepare) Task=`297526c6-826f-4028-a450-6e9279a17547`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:49,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`297526c6-826f-4028-a450-6e9279a17547`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:49,834::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:49,835::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:49,835::task::993::Storage.TaskManager.Task::(_decref) Task=`297526c6-826f-4028-a450-6e9279a17547`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:56:49,868::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:56:49,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`f503d883-8e63-4215-969b-42d8d9222258`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:56:49,868::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:56:49,869::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8e451ae0-eb0d-4f35-9f85-a5b62b41c10f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:56:49,869::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:56:49,870::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:56:49,870::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8e451ae0-eb0d-4f35-9f85-a5b62b41c10f`::Granted request >Thread-46::DEBUG::2014-08-22 09:56:49,870::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f503d883-8e63-4215-969b-42d8d9222258`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:56:49,870::task::993::Storage.TaskManager.Task::(_decref) Task=`f503d883-8e63-4215-969b-42d8d9222258`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:56:49,872::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:49,872::task::1191::Storage.TaskManager.Task::(prepare) Task=`f503d883-8e63-4215-969b-42d8d9222258`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:49,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`f503d883-8e63-4215-969b-42d8d9222258`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:56:49,872::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:56:49,872::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:56:49,872::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:56:49,872::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:56:49,873::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:56:49,873::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:56:49,873::task::993::Storage.TaskManager.Task::(_decref) Task=`f503d883-8e63-4215-969b-42d8d9222258`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:56:50,781::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:50,781::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:50,781::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:50,781::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:50,781::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:50,782::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:50,782::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:50,782::__init__::225::IOProcess::(_processLogs) (1814) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:50,782::__init__::225::IOProcess::(_processLogs) (1814) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:50,782::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1814) Building response >Thread-17::DEBUG::2014-08-22 09:56:50,783::__init__::225::IOProcess::(_processLogs) (1814) Queuing response >Thread-22::DEBUG::2014-08-22 09:56:50,783::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:50,784::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:50,790::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:56:50,791::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000215901 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:50,791::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) (1815) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:50,792::__init__::225::IOProcess::(_processLogs) (1815) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:50,793::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1815) Building response >Thread-17::DEBUG::2014-08-22 09:56:50,793::__init__::225::IOProcess::(_processLogs) (1815) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:50,794::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:50,794::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:50,794::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:50,794::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:56:50,794::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:50,794::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) (1816) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) (1816) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) (1816) Building response >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) (1816) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:50,795::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:50,796::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:50,796::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:56:50,796::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:50,796::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:50,796::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) (1817) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) (1817) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) (1817) Building response >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) (1817) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:56:50,797::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:50,798::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:50,798::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:50,798::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) (1818) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) (1818) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) (1818) Building response >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) (1818) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:50,799::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:56:51,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:51,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120252 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:53,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:53,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116152 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:55,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:55,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116494 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:56:57,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:57,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113925 s, 89.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:59,543::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:59,543::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:59,543::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:59,543::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:56:59,544::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:56:59,544::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:59,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:56:59,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000150542 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:56:59,552::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:59,553::__init__::225::IOProcess::(_processLogs) (1819) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:59,553::__init__::225::IOProcess::(_processLogs) (1819) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:59,553::__init__::225::IOProcess::(_processLogs) (1819) Building response >Thread-17::DEBUG::2014-08-22 09:56:59,553::__init__::225::IOProcess::(_processLogs) (1819) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:59,553::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:59,553::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:56:59,554::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:56:59,554::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:56:59,554::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:56:59,554::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:56:59,554::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:56:59,555::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:56:59,555::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:56:59,555::__init__::225::IOProcess::(_processLogs) (1820) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:56:59,556::__init__::225::IOProcess::(_processLogs) (1820) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:56:59,556::__init__::225::IOProcess::(_processLogs) (1820) Building response >Thread-17::DEBUG::2014-08-22 09:56:59,556::__init__::225::IOProcess::(_processLogs) (1820) Queuing response >Thread-17::DEBUG::2014-08-22 09:56:59,556::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:56:59,556::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:56:59,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:56:59,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115999 s, 88.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:56:59,924::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:56:59,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`8dbf56a9-93d7-476a-9534-20c13dec070c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:56:59,924::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:56:59,924::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:59,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`8dbf56a9-93d7-476a-9534-20c13dec070c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:56:59,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`8dbf56a9-93d7-476a-9534-20c13dec070c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:56:59,925::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:56:59,925::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:56:59,925::task::993::Storage.TaskManager.Task::(_decref) Task=`8dbf56a9-93d7-476a-9534-20c13dec070c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:56:59,954::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:56:59,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b6e397a-3fed-4a4e-a30a-883c3af33774`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:56:59,954::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:56:59,954::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e9248391-bbc3-4617-94eb-0e52c55679c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:56:59,955::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:56:59,955::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:56:59,955::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e9248391-bbc3-4617-94eb-0e52c55679c0`::Granted request >Thread-46::DEBUG::2014-08-22 09:56:59,955::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1b6e397a-3fed-4a4e-a30a-883c3af33774`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:56:59,955::task::993::Storage.TaskManager.Task::(_decref) Task=`1b6e397a-3fed-4a4e-a30a-883c3af33774`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:56:59,957::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:59,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b6e397a-3fed-4a4e-a30a-883c3af33774`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:56:59,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b6e397a-3fed-4a4e-a30a-883c3af33774`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:56:59,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:56:59,957::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:56:59,958::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:56:59,958::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:56:59,958::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:56:59,958::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:56:59,958::task::993::Storage.TaskManager.Task::(_decref) Task=`1b6e397a-3fed-4a4e-a30a-883c3af33774`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:57:00,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`cea25462-7e58-4302-a4f4-83421e7a3506`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:00,497::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:57:00,498::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000150542', 'lastCheck': '0.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000215901', 'lastCheck': '9.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:00,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`cea25462-7e58-4302-a4f4-83421e7a3506`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000150542', 'lastCheck': '0.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000215901', 'lastCheck': '9.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:00,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`cea25462-7e58-4302-a4f4-83421e7a3506`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:00,498::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:00,498::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:00,498::task::993::Storage.TaskManager.Task::(_decref) Task=`cea25462-7e58-4302-a4f4-83421e7a3506`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:57:00,799::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:00,799::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:00,799::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:00,799::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:00,799::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:00,799::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) (1821) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) (1821) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) (1821) Building response >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) (1821) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:00,800::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:57:00,801::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:57:00,807::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223803 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:00,808::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:00,808::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:00,809::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:00,809::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:00,809::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:00,809::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:00,809::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:00,809::__init__::225::IOProcess::(_processLogs) (1822) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) (1822) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) (1822) Building response >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) (1822) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:57:00,810::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) (1823) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) (1823) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) (1823) Building response >Thread-17::DEBUG::2014-08-22 09:57:00,811::__init__::225::IOProcess::(_processLogs) (1823) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:00,812::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:00,812::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:00,812::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:00,812::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:57:00,812::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:00,812::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) (1824) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) (1824) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) (1824) Building response >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) (1824) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:00,813::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:00,814::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:00,814::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:57:00,814::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:00,814::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:00,814::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) (1825) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) (1825) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) (1825) Building response >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) (1825) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:00,815::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:57:01,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:01,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116688 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:03,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:03,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114829 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:05,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:05,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114744 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:07,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:07,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114447 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:09,556::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:09,556::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:09,556::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:09,557::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:09,557::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 09:57:09,557::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:09,558::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:09,564::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:57:09,565::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000152078 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:09,565::__init__::225::IOProcess::(_processLogs) (1826) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:09,566::__init__::225::IOProcess::(_processLogs) (1826) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:09,566::__init__::225::IOProcess::(_processLogs) (1826) Building response >Thread-17::DEBUG::2014-08-22 09:57:09,566::__init__::225::IOProcess::(_processLogs) (1826) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:09,566::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:09,567::__init__::225::IOProcess::(_processLogs) (1827) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:09,568::__init__::225::IOProcess::(_processLogs) (1827) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:09,568::__init__::225::IOProcess::(_processLogs) (1827) Building response >Thread-17::DEBUG::2014-08-22 09:57:09,568::__init__::225::IOProcess::(_processLogs) (1827) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:09,568::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:09,569::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:57:09,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:09,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114777 s, 89.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:57:10,040::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:57:10,041::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a343591-9a24-41c3-9620-111d5c6aa524`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:10,041::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:57:10,041::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:10,041::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a343591-9a24-41c3-9620-111d5c6aa524`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:10,041::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a343591-9a24-41c3-9620-111d5c6aa524`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:10,041::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:10,041::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:10,041::task::993::Storage.TaskManager.Task::(_decref) Task=`8a343591-9a24-41c3-9620-111d5c6aa524`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:57:10,075::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:57:10,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0cb5430-9b12-4981-b6c6-3297c0afc69f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:57:10,076::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:57:10,076::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f5e5a284-1b49-4319-82fc-0c0b13af3dc7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:57:10,076::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:57:10,076::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:57:10,076::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f5e5a284-1b49-4319-82fc-0c0b13af3dc7`::Granted request >Thread-46::DEBUG::2014-08-22 09:57:10,076::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f0cb5430-9b12-4981-b6c6-3297c0afc69f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:57:10,077::task::993::Storage.TaskManager.Task::(_decref) Task=`f0cb5430-9b12-4981-b6c6-3297c0afc69f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:57:10,078::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:10,079::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0cb5430-9b12-4981-b6c6-3297c0afc69f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:10,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0cb5430-9b12-4981-b6c6-3297c0afc69f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:57:10,079::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:57:10,079::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:57:10,079::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:57:10,079::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:57:10,079::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:57:10,079::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:57:10,079::task::993::Storage.TaskManager.Task::(_decref) Task=`f0cb5430-9b12-4981-b6c6-3297c0afc69f`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:57:10,815::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:10,815::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:10,815::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:10,816::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:57:10,817::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:10,817::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:10,823::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:57:10,824::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000157357 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:10,824::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:10,825::__init__::225::IOProcess::(_processLogs) (1828) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:10,825::__init__::225::IOProcess::(_processLogs) (1828) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:10,825::__init__::225::IOProcess::(_processLogs) (1828) Building response >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) (1828) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:10,826::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:10,827::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:10,827::__init__::225::IOProcess::(_processLogs) (1829) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:10,827::__init__::225::IOProcess::(_processLogs) (1829) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:10,827::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(1829) Building response >Thread-17::DEBUG::2014-08-22 09:57:10,828::__init__::225::IOProcess::(_processLogs) (1829) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:10,828::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:10,828::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:10,828::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:10,828::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:57:10,828::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:10,829::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:10,829::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:10,829::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:10,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:10,829::__init__::225::IOProcess::(_processLogs) (1830) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:10,829::__init__::225::IOProcess::(_processLogs) (1830) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) (1830) Building response >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) (1830) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:57:10,830::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) (1831) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) (1831) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) (1831) Building response >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) (1831) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:10,831::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:10,832::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:10,832::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:57:10,832::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:10,832::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:10,832::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) (1832) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) (1832) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) (1832) Building response >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) (1832) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:10,833::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:57:11,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:11,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118509 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:13,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:13,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115442 s, 88.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:57:15,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`65113993-0441-4f66-9112-dee2d7280040`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:15,668::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:57:15,669::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000152078', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000157357', 'lastCheck': '4.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:15,669::task::1191::Storage.TaskManager.Task::(prepare) Task=`65113993-0441-4f66-9112-dee2d7280040`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000152078', 'lastCheck': '6.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000157357', 'lastCheck': '4.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:15,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`65113993-0441-4f66-9112-dee2d7280040`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:15,669::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:15,669::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:15,669::task::993::Storage.TaskManager.Task::(_decref) Task=`65113993-0441-4f66-9112-dee2d7280040`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:57:15,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:16,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131234 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:18,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:18,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011721 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) (1833) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:19,570::__init__::225::IOProcess::(_processLogs) (1833) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:19,571::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|(1833) Building response >Thread-21::DEBUG::2014-08-22 09:57:19,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:19,572::__init__::225::IOProcess::(_processLogs) (1833) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:19,578::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:57:19,579::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000243546 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:19,580::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:19,580::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:19,580::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) (1834) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) (1834) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) (1834) Building response >Thread-17::DEBUG::2014-08-22 09:57:19,581::__init__::225::IOProcess::(_processLogs) (1834) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:19,582::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:57:19,583::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:57:20,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:20,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0127014 s, 80.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:57:20,244::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:57:20,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`63f0cabf-0a24-41d0-9a17-e8758acfd257`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:20,244::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:57:20,245::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:20,245::task::1191::Storage.TaskManager.Task::(prepare) Task=`63f0cabf-0a24-41d0-9a17-e8758acfd257`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:20,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`63f0cabf-0a24-41d0-9a17-e8758acfd257`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:20,245::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:20,245::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:20,245::task::993::Storage.TaskManager.Task::(_decref) Task=`63f0cabf-0a24-41d0-9a17-e8758acfd257`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:57:20,288::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:57:20,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`41ebe09a-a03a-440b-a650-633d60250a91`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:57:20,288::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:57:20,289::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4ce08e9c-d336-40f4-9408-2f9059d258b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:57:20,289::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:57:20,289::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:57:20,289::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4ce08e9c-d336-40f4-9408-2f9059d258b5`::Granted request >Thread-46::DEBUG::2014-08-22 09:57:20,289::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`41ebe09a-a03a-440b-a650-633d60250a91`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:57:20,289::task::993::Storage.TaskManager.Task::(_decref) Task=`41ebe09a-a03a-440b-a650-633d60250a91`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:57:20,291::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:20,291::task::1191::Storage.TaskManager.Task::(prepare) Task=`41ebe09a-a03a-440b-a650-633d60250a91`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:20,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`41ebe09a-a03a-440b-a650-633d60250a91`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:57:20,292::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:57:20,292::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:57:20,292::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:57:20,292::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:57:20,292::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:57:20,292::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:57:20,292::task::993::Storage.TaskManager.Task::(_decref) Task=`41ebe09a-a03a-440b-a650-633d60250a91`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:57:20,833::__init__::225::IOProcess::(_processLogs) Generating json...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:20,833::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:20,835::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-22::DEBUG::2014-08-22 09:57:20,835::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:20,835::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:20,842::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:57:20,843::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000145819 s, 3.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:20,843::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) (1835) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) (1835) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) (1835) Building response >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) (1835) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:20,844::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) (1836) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:20,845::__init__::225::IOProcess::(_processLogs) (1836) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:20,846::__init__::225::IOProcess::(_processLogs) (1836) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:20,846::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:20,846::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) (1837) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:20,847::__init__::225::IOProcess::(_processLogs) (1837) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:20,848::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1837) Building response >Thread-17::DEBUG::2014-08-22 09:57:20,848::__init__::225::IOProcess::(_processLogs) (1837) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:20,848::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:20,848::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:20,849::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:20,849::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:57:20,849::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:20,849::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) (1838) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) (1838) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) (1838) Building response >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) (1838) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:20,850::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:57:20,851::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:20,851::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:20,851::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) (1839) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) (1839) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) (1839) Building response >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) (1839) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:20,852::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:57:22,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:22,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129041 s, 79.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:24,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:24,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114777 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:26,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:26,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012808 s, 79.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:28,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:28,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118724 s, 86.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:29,583::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:29,583::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:29,583::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:29,584::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:57:29,585::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:29,585::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:29,591::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:29,591::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:29,592::__init__::225::IOProcess::(_processLogs) (1840) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:29,592::__init__::225::IOProcess::(_processLogs) (1840) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:29,592::__init__::225::IOProcess::(_processLogs) (1840) Building response >Thread-17::DEBUG::2014-08-22 09:57:29,592::__init__::225::IOProcess::(_processLogs) (1840) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:29,592::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:29,592::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:57:29,593::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00027782 s, 1.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:29,593::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:29,593::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:29,593::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:29,594::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:29,594::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:29,594::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:29,594::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:29,594::__init__::225::IOProcess::(_processLogs) (1841) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:29,594::__init__::225::IOProcess::(_processLogs) (1841) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:29,595::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1841) Building response >Thread-17::DEBUG::2014-08-22 09:57:29,595::__init__::225::IOProcess::(_processLogs) (1841) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:29,596::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:29,596::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:57:30,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:30,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118026 s, 86.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:57:30,390::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:57:30,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f22fccc-d45d-4e02-8180-18ae2e989029`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:30,391::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:57:30,392::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:30,392::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f22fccc-d45d-4e02-8180-18ae2e989029`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:30,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f22fccc-d45d-4e02-8180-18ae2e989029`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:30,393::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:30,393::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:30,394::task::993::Storage.TaskManager.Task::(_decref) Task=`6f22fccc-d45d-4e02-8180-18ae2e989029`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:57:30,424::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:57:30,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`ace38e7b-2c54-45a2-9ace-17fbc6cbd7b4`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:57:30,424::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:57:30,425::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e3e412e0-415d-4adc-baef-2c2f02873389`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:57:30,425::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:57:30,425::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:57:30,425::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e3e412e0-415d-4adc-baef-2c2f02873389`::Granted request >Thread-46::DEBUG::2014-08-22 09:57:30,425::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ace38e7b-2c54-45a2-9ace-17fbc6cbd7b4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:57:30,426::task::993::Storage.TaskManager.Task::(_decref) Task=`ace38e7b-2c54-45a2-9ace-17fbc6cbd7b4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:57:30,428::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:30,428::task::1191::Storage.TaskManager.Task::(prepare) Task=`ace38e7b-2c54-45a2-9ace-17fbc6cbd7b4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:30,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`ace38e7b-2c54-45a2-9ace-17fbc6cbd7b4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:57:30,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:57:30,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:57:30,428::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:57:30,428::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:57:30,428::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:57:30,428::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:57:30,428::task::993::Storage.TaskManager.Task::(_decref) Task=`ace38e7b-2c54-45a2-9ace-17fbc6cbd7b4`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:57:30,852::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:30,852::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:30,852::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:30,852::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:30,853::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:30,853::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:30,853::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:30,853::__init__::225::IOProcess::(_processLogs) (1842) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:30,853::__init__::225::IOProcess::(_processLogs) (1842) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:30,853::__init__::225::IOProcess::(_processLogs) (1842) Queuing response >Thread-22::DEBUG::2014-08-22 09:57:30,854::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:30,854::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:30,861::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:57:30,861::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000209605 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:30,862::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:30,862::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:30,862::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:30,862::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:30,862::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:30,862::__init__::225::IOProcess::(_processLogs) (1843) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:30,863::__init__::225::IOProcess::(_processLogs) (1843) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:30,863::__init__::225::IOProcess::(_processLogs) (1843) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:30,864::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:30,864::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:30,864::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:57:30,864::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) (1844) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) (1844) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) (1844) Building response >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) (1844) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:30,865::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:30,866::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:30,866::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:57:30,866::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:30,866::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:30,866::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:30,867::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:30,867::__init__::225::IOProcess::(_processLogs) (1845) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:30,867::__init__::225::IOProcess::(_processLogs) (1845) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:30,867::__init__::225::IOProcess::(_processLogs) (1845) Building response >Thread-17::DEBUG::2014-08-22 09:57:30,867::__init__::225::IOProcess::(_processLogs) (1845) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:30,867::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:30,868::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:30,868::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:57:30,868::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:30,868::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:30,869::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:30,869::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:30,869::__init__::225::IOProcess::(_processLogs) (1846) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:30,869::__init__::225::IOProcess::(_processLogs) (1846) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:30,869::__init__::225::IOProcess::(_processLogs) (1846) Building response >Thread-17::DEBUG::2014-08-22 09:57:30,870::__init__::225::IOProcess::(_processLogs) (1846) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:30,870::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:30,870::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:57:30,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3875fec-dc31-4c1a-8392-a164c8478744`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:30,937::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:57:30,937::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00027782', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209605', 'lastCheck': '0.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:30,937::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3875fec-dc31-4c1a-8392-a164c8478744`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00027782', 'lastCheck': '1.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209605', 'lastCheck': '0.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:30,937::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3875fec-dc31-4c1a-8392-a164c8478744`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:30,937::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:30,937::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:30,937::task::993::Storage.TaskManager.Task::(_decref) Task=`a3875fec-dc31-4c1a-8392-a164c8478744`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:57:32,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:32,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206483 s, 49.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:34,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:34,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113367 s, 90.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:36,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:36,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113473 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:38,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:38,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180727 s, 56.7 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 09:57:39,596::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:57:39,598::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:57:39,598::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,598::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:39,598::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,598::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,598::__init__::225::IOProcess::(_processLogs) (1847) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:57:39,599::__init__::225::IOProcess::(_processLogs) (1847) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:57:39,600::__init__::225::IOProcess::(_processLogs) (1847) Building response >Thread-17::DEBUG::2014-08-22 09:57:39,600::__init__::225::IOProcess::(_processLogs) (1847) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:39,601::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:39,602::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:57:39,602::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:39,603::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:57:39,604::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,604::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:39,605::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,606::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,607::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 09:57:39,608::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 09:57:39,608::__init__::225::IOProcess::(_processLogs) (1848) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:57:39,609::__init__::225::IOProcess::(_processLogs) (1848) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:57:39,609::__init__::225::IOProcess::(_processLogs) (1848) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:39,609::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) (1849) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:57:39,610::__init__::225::IOProcess::(_processLogs) (1849) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:57:39,639::__init__::225::IOProcess::(_processLogs) (1849) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:39,640::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-21::DEBUG::2014-08-22 09:57:39,640::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:57:39,641::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 09:57:39,641::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,641::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,641::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:39,641::__init__::225::IOProcess::(_processLogs) (1850) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:57:39,642::__init__::225::IOProcess::(_processLogs) (1850) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:57:39,642::__init__::225::IOProcess::(_processLogs) (1850) Queuing response >Thread-21::DEBUG::2014-08-22 09:57:39,642::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 09:57:39,643::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:39,644::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 09:57:39,644::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 09:57:39,644::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,644::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) (1851) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) (1851) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) (1851) Building response >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) (1851) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:39,645::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) (1852) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) (1852) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 09:57:39,646::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:57:39,646::__init__::225::IOProcess::(_processLogs) (1852) Queuing response >Thread-21::INFO::2014-08-22 09:57:39,647::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:57:39,647::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 09:57:39,648::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:57:39,648::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) (1853) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:39,649::__init__::225::IOProcess::(_processLogs) (1853) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:39,650::__init__::225::IOProcess::(_processLogs) (1853) Queuing response >Thread-21::DEBUG::2014-08-22 09:57:39,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:39,651::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:39,657::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:57:39,658::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000252606 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) (1854) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:39,659::__init__::225::IOProcess::(_processLogs) (1854) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:39,660::__init__::225::IOProcess::(_processLogs) (1854) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:39,661::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:39,661::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:57:40,485::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:57:40,485::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d250ea3-c6ee-413d-bb7d-ab45e34645b2`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:40,485::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:57:40,485::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:40,485::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d250ea3-c6ee-413d-bb7d-ab45e34645b2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:40,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d250ea3-c6ee-413d-bb7d-ab45e34645b2`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:40,486::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:40,486::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:40,486::task::993::Storage.TaskManager.Task::(_decref) Task=`8d250ea3-c6ee-413d-bb7d-ab45e34645b2`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:57:40,525::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:57:40,525::task::595::Storage.TaskManager.Task::(_updateState) Task=`bddf5e74-e164-427e-a3ca-865b6fe33d49`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:57:40,525::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:57:40,526::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23910626-e6a1-4f66-8fc4-d12e6c583414`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:57:40,526::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:57:40,526::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:57:40,526::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`23910626-e6a1-4f66-8fc4-d12e6c583414`::Granted request >Thread-46::DEBUG::2014-08-22 09:57:40,526::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bddf5e74-e164-427e-a3ca-865b6fe33d49`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:57:40,526::task::993::Storage.TaskManager.Task::(_decref) Task=`bddf5e74-e164-427e-a3ca-865b6fe33d49`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:57:40,528::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:40,528::task::1191::Storage.TaskManager.Task::(prepare) Task=`bddf5e74-e164-427e-a3ca-865b6fe33d49`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:40,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`bddf5e74-e164-427e-a3ca-865b6fe33d49`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:57:40,529::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:57:40,529::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:57:40,529::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:57:40,529::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:57:40,529::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:57:40,529::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:57:40,529::task::993::Storage.TaskManager.Task::(_decref) Task=`bddf5e74-e164-427e-a3ca-865b6fe33d49`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:57:40,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:40,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117761 s, 87.0 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-08-22 09:57:40,869::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:57:40,871::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 09:57:40,871::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,871::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,871::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,871::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,872::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,872::__init__::225::IOProcess::(_processLogs) (1855) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:57:40,872::__init__::225::IOProcess::(_processLogs) (1855) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:57:40,872::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1855) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) (1855) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,873::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) (1856) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) (1856) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) (1856) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) (1856) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,874::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 09:57:40,876::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) (1857) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 09:57:40,877::__init__::225::IOProcess::(_processLogs) (1857) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 09:57:40,890::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1857) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,890::__init__::225::IOProcess::(_processLogs) (1857) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,890::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 09:57:40,891::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 09:57:40,891::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:40,891::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) (1858) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) (1858) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 09:57:40,892::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1858) Building response >Thread-22::DEBUG::2014-08-22 09:57:40,893::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 09:57:40,893::__init__::225::IOProcess::(_processLogs) (1858) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,894::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,894::__init__::225::IOProcess::(_processLogs) Sending response sized 832 >Thread-17::DEBUG::2014-08-22 09:57:40,894::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,894::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 09:57:40,894::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) (1859) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) (1859) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) (1859) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) (1859) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,895::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) (1860) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 09:57:40,896::__init__::225::IOProcess::(_processLogs) (1860) Got request for method 'glob' >Thread-22::DEBUG::2014-08-22 09:57:40,897::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 09:57:40,897::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1860) Building response >Thread-22::INFO::2014-08-22 09:57:40,897::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 09:57:40,897::__init__::225::IOProcess::(_processLogs) (1860) Queuing response >Thread-22::INFO::2014-08-22 09:57:40,898::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 09:57:40,898::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,898::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 09:57:40,899::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,899::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:40,899::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,899::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:57:40,900::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:40,900::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,906::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:57:40,907::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000219908 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:40,907::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,908::__init__::225::IOProcess::(_processLogs) (1861) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:40,908::__init__::225::IOProcess::(_processLogs) (1861) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:40,908::__init__::225::IOProcess::(_processLogs) (1861) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) (1861) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,909::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,910::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,910::__init__::225::IOProcess::(_processLogs) (1862) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:40,910::__init__::225::IOProcess::(_processLogs) (1862) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:40,910::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1862) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,911::__init__::225::IOProcess::(_processLogs) (1862) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,911::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,911::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:40,911::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,911::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:57:40,912::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,912::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,912::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,912::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,912::__init__::225::IOProcess::(_processLogs) (1863) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) (1863) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) (1863) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) (1863) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:57:40,913::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) (1864) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) (1864) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) (1864) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) (1864) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,914::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,915::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:40,915::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:40,915::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:57:40,915::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:40,915::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) (1865) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) (1865) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) (1865) Building response >Thread-17::DEBUG::2014-08-22 09:57:40,916::__init__::225::IOProcess::(_processLogs) (1865) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:40,917::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:40,917::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:57:42,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:42,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0127105 s, 80.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:44,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:44,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117806 s, 86.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:57:46,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`912f041f-49a8-4a2b-9f0f-0ca3cb914519`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:46,136::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:57:46,137::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252606', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219908', 'lastCheck': '5.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:46,137::task::1191::Storage.TaskManager.Task::(prepare) Task=`912f041f-49a8-4a2b-9f0f-0ca3cb914519`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252606', 'lastCheck': '6.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219908', 'lastCheck': '5.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:57:46,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`912f041f-49a8-4a2b-9f0f-0ca3cb914519`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:46,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:46,137::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:46,137::task::993::Storage.TaskManager.Task::(_decref) Task=`912f041f-49a8-4a2b-9f0f-0ca3cb914519`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:57:46,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:46,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116787 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:48,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:48,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120394 s, 85.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:49,661::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:49,661::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) (1866) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:49,662::__init__::225::IOProcess::(_processLogs) (1866) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:49,663::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1866) Building response >Thread-21::DEBUG::2014-08-22 09:57:49,663::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:49,664::__init__::225::IOProcess::(_processLogs) (1866) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:49,670::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:57:49,671::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000221069 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:49,671::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:49,672::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:49,672::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:49,672::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) (1867) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) (1867) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) (1867) Building response >Thread-17::DEBUG::2014-08-22 09:57:49,673::__init__::225::IOProcess::(_processLogs) (1867) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:49,674::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:57:49,675::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:57:50,607::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:57:50,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`f848b7fc-f9b6-40d8-8184-133b7a5c9a64`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:57:50,607::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:57:50,607::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:50,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`f848b7fc-f9b6-40d8-8184-133b7a5c9a64`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:57:50,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`f848b7fc-f9b6-40d8-8184-133b7a5c9a64`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:57:50,608::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:57:50,608::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:57:50,608::task::993::Storage.TaskManager.Task::(_decref) Task=`f848b7fc-f9b6-40d8-8184-133b7a5c9a64`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:57:50,641::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:57:50,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`9726e57d-38c3-4032-a038-ff2d31f177a9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:57:50,641::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:57:50,641::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`55a233e8-7908-4a2f-8cd5-181c1c993e98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:57:50,641::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:57:50,641::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:57:50,642::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`55a233e8-7908-4a2f-8cd5-181c1c993e98`::Granted request >Thread-46::DEBUG::2014-08-22 09:57:50,642::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9726e57d-38c3-4032-a038-ff2d31f177a9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:57:50,642::task::993::Storage.TaskManager.Task::(_decref) Task=`9726e57d-38c3-4032-a038-ff2d31f177a9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:57:50,644::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:50,644::task::1191::Storage.TaskManager.Task::(prepare) Task=`9726e57d-38c3-4032-a038-ff2d31f177a9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:57:50,644::task::595::Storage.TaskManager.Task::(_updateState) Task=`9726e57d-38c3-4032-a038-ff2d31f177a9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:57:50,644::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:57:50,644::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:57:50,644::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:57:50,644::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:57:50,644::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:57:50,644::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:57:50,645::task::993::Storage.TaskManager.Task::(_decref) Task=`9726e57d-38c3-4032-a038-ff2d31f177a9`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:57:50,916::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:50,916::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:50,917::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:50,917::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:57:50,918::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:50,918::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:50,924::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:57:50,925::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000244898 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:50,925::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) (1868) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) (1868) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) (1868) Building response >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) (1868) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:57:50,927::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:50,928::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:50,928::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:50,928::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:50,928::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:50,928::__init__::225::IOProcess::(_processLogs) (1869) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:50,928::__init__::225::IOProcess::(_processLogs) (1869) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:50,929::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1869) Building response >Thread-17::DEBUG::2014-08-22 09:57:50,929::__init__::225::IOProcess::(_processLogs) (1869) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:50,929::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:50,929::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:50,930::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:50,930::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:57:50,930::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:50,930::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:50,930::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) (1870) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) (1870) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) (1870) Building response >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) (1870) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:57:50,931::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:50,932::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:50,932::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:50,932::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:50,932::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:50,932::__init__::225::IOProcess::(_processLogs) (1871) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:50,932::__init__::225::IOProcess::(_processLogs) (1871) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) (1871) Building response >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) (1871) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:57:50,933::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:50,934::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:50,934::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:50,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:50,934::__init__::225::IOProcess::(_processLogs) (1872) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:57:50,934::__init__::225::IOProcess::(_processLogs) (1872) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:57:50,934::__init__::225::IOProcess::(_processLogs) (1872) Building response >Thread-17::DEBUG::2014-08-22 09:57:50,935::__init__::225::IOProcess::(_processLogs) (1872) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:50,935::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:50,935::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:57:50,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:50,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118423 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:53,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:53,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116521 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:55,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:55,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114841 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:57,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:57,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114776 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:57:59,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:57:59,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114949 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:59,675::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:59,675::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:59,675::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:59,676::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:57:59,677::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:57:59,677::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:59,683::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:57:59,684::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000173207 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:57:59,684::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:59,685::__init__::225::IOProcess::(_processLogs) (1873) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:59,685::__init__::225::IOProcess::(_processLogs) (1873) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:59,685::__init__::225::IOProcess::(_processLogs) (1873) Building response >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) (1873) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:57:59,686::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:57:59,687::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:57:59,687::__init__::225::IOProcess::(_processLogs) (1874) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:57:59,687::__init__::225::IOProcess::(_processLogs) (1874) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:57:59,687::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1874) Building response >Thread-17::DEBUG::2014-08-22 09:57:59,688::__init__::225::IOProcess::(_processLogs) (1874) Queuing response >Thread-17::DEBUG::2014-08-22 09:57:59,688::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:57:59,688::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:58:00,694::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-47::DEBUG::2014-08-22 09:58:00,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b657920-f8b9-42aa-ab11-90f283ebe447`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:00,694::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:58:00,695::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:00,695::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b657920-f8b9-42aa-ab11-90f283ebe447`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:00,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b657920-f8b9-42aa-ab11-90f283ebe447`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:00,695::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:00,695::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:00,695::task::993::Storage.TaskManager.Task::(_decref) Task=`1b657920-f8b9-42aa-ab11-90f283ebe447`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:58:00,761::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-46::DEBUG::2014-08-22 09:58:00,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`f32bb0ce-2ee6-4670-9b2c-af830f4d48c8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:58:00,761::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:58:00,762::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1f0da0b9-6b4e-48e5-b0fd-9c16d17ba46d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:58:00,762::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:58:00,762::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:58:00,762::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1f0da0b9-6b4e-48e5-b0fd-9c16d17ba46d`::Granted request >Thread-46::DEBUG::2014-08-22 09:58:00,763::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f32bb0ce-2ee6-4670-9b2c-af830f4d48c8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:58:00,763::task::993::Storage.TaskManager.Task::(_decref) Task=`f32bb0ce-2ee6-4670-9b2c-af830f4d48c8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:58:00,765::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:00,765::task::1191::Storage.TaskManager.Task::(prepare) Task=`f32bb0ce-2ee6-4670-9b2c-af830f4d48c8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:00,765::task::595::Storage.TaskManager.Task::(_updateState) Task=`f32bb0ce-2ee6-4670-9b2c-af830f4d48c8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:58:00,765::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:58:00,765::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:58:00,766::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:58:00,766::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:58:00,766::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:58:00,766::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:58:00,766::task::993::Storage.TaskManager.Task::(_decref) Task=`f32bb0ce-2ee6-4670-9b2c-af830f4d48c8`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:58:00,934::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:00,935::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:00,935::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:00,935::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:00,936::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 09:58:00,936::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:00,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:00,943::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 09:58:00,943::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000303841 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:00,944::__init__::225::IOProcess::(_processLogs) (1875) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:00,945::__init__::225::IOProcess::(_processLogs) (1875) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:00,945::__init__::225::IOProcess::(_processLogs) (1875) Building response >Thread-17::DEBUG::2014-08-22 09:58:00,945::__init__::225::IOProcess::(_processLogs) (1875) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:00,945::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:00,945::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:00,946::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:00,946::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:00,946::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:00,947::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) (1876) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) (1876) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) (1876) Building response >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) (1876) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:00,948::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:00,949::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:58:00,949::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:00,949::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:00,949::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) (1877) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) (1877) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) (1877) Building response >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) (1877) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:00,950::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) (1878) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) (1878) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:00,951::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1878) Building response >Thread-17::DEBUG::2014-08-22 09:58:00,952::__init__::225::IOProcess::(_processLogs) (1878) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:00,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:00,952::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:00,952::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:00,952::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:58:00,952::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:00,953::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:00,953::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:00,953::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:00,953::__init__::225::IOProcess::(_processLogs) (1879) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:00,954::__init__::225::IOProcess::(_processLogs) (1879) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:00,954::__init__::225::IOProcess::(_processLogs) (1879) Building response >Thread-17::DEBUG::2014-08-22 09:58:00,954::__init__::225::IOProcess::(_processLogs) (1879) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:00,954::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:00,954::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:58:01,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:01,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119627 s, 85.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:58:01,331::task::595::Storage.TaskManager.Task::(_updateState) Task=`239e50d0-5119-4e32-91db-e7f13d02ee21`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:01,331::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:58:01,331::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000173207', 'lastCheck': '1.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000303841', 'lastCheck': '0.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:01,331::task::1191::Storage.TaskManager.Task::(prepare) Task=`239e50d0-5119-4e32-91db-e7f13d02ee21`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000173207', 'lastCheck': '1.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000303841', 'lastCheck': '0.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:01,331::task::595::Storage.TaskManager.Task::(_updateState) Task=`239e50d0-5119-4e32-91db-e7f13d02ee21`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:01,331::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:01,331::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:01,331::task::993::Storage.TaskManager.Task::(_decref) Task=`239e50d0-5119-4e32-91db-e7f13d02ee21`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:03,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:03,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113386 s, 90.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:05,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:05,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116829 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:07,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:07,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113869 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:09,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:09,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152138 s, 67.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:09,689::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:09,689::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:09,689::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:09,689::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:58:09,690::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:09,690::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:09,696::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:09,696::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:09,697::__init__::225::IOProcess::(_processLogs) (1880) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:09,697::__init__::225::IOProcess::(_processLogs) (1880) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:09,697::__init__::225::IOProcess::(_processLogs) (1880) Building response >Thread-17::DEBUG::2014-08-22 09:58:09,697::__init__::225::IOProcess::(_processLogs) (1880) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:09,697::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:09,697::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:58:09,698::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182142 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:09,698::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:09,698::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:09,698::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:09,698::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:09,699::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:09,699::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:09,699::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:09,699::__init__::225::IOProcess::(_processLogs) (1881) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:09,699::__init__::225::IOProcess::(_processLogs) (1881) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:09,700::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1881) Building response >Thread-17::DEBUG::2014-08-22 09:58:09,700::__init__::225::IOProcess::(_processLogs) (1881) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:09,701::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:09,701::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:58:10,813::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:58:10,814::task::595::Storage.TaskManager.Task::(_updateState) Task=`691828ec-00ac-4a3f-9d92-e3b0a5f9ca15`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:10,814::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:58:10,814::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:10,814::task::1191::Storage.TaskManager.Task::(prepare) Task=`691828ec-00ac-4a3f-9d92-e3b0a5f9ca15`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:10,814::task::595::Storage.TaskManager.Task::(_updateState) Task=`691828ec-00ac-4a3f-9d92-e3b0a5f9ca15`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:10,814::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:10,814::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:10,814::task::993::Storage.TaskManager.Task::(_decref) Task=`691828ec-00ac-4a3f-9d92-e3b0a5f9ca15`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:58:10,873::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:58:10,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`98d68d6b-b50d-40f9-bc6e-7d243524c2b3`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:58:10,873::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:58:10,873::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b3329274-026f-4022-900d-0843c7c3f166`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:58:10,873::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:58:10,873::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:58:10,873::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b3329274-026f-4022-900d-0843c7c3f166`::Granted request >Thread-46::DEBUG::2014-08-22 09:58:10,874::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`98d68d6b-b50d-40f9-bc6e-7d243524c2b3`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:58:10,874::task::993::Storage.TaskManager.Task::(_decref) Task=`98d68d6b-b50d-40f9-bc6e-7d243524c2b3`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:58:10,876::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:10,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`98d68d6b-b50d-40f9-bc6e-7d243524c2b3`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:10,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`98d68d6b-b50d-40f9-bc6e-7d243524c2b3`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:58:10,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:58:10,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:58:10,876::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:58:10,876::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:58:10,876::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:58:10,876::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:58:10,877::task::993::Storage.TaskManager.Task::(_decref) Task=`98d68d6b-b50d-40f9-bc6e-7d243524c2b3`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:10,954::__init__::225::IOProcess::(_processLogs) (1882) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:10,955::__init__::225::IOProcess::(_processLogs) (1882) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:10,955::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1882) Building response >Thread-17::DEBUG::2014-08-22 09:58:10,955::__init__::225::IOProcess::(_processLogs) (1882) Queuing response >Thread-22::DEBUG::2014-08-22 09:58:10,956::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:10,956::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:10,962::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:58:10,963::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000205184 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:10,964::__init__::225::IOProcess::(_processLogs) (1883) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:10,965::__init__::225::IOProcess::(_processLogs) (1883) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:10,965::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1883) Building response >Thread-17::DEBUG::2014-08-22 09:58:10,966::__init__::225::IOProcess::(_processLogs) (1883) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:10,966::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:10,966::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:10,966::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:10,966::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:58:10,966::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:10,967::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:10,967::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:10,967::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:10,967::__init__::225::IOProcess::(_processLogs) (1884) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:10,967::__init__::225::IOProcess::(_processLogs) (1884) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:10,967::__init__::225::IOProcess::(_processLogs) (1884) Building response >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) (1884) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:10,968::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) (1885) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) (1885) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) (1885) Building response >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) (1885) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:10,969::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:10,970::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:10,970::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:58:10,970::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:10,970::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:10,970::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) (1886) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) (1886) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) (1886) Building response >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) (1886) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:10,971::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:58:11,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:11,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141103 s, 72.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:13,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:13,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115817 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:15,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:15,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114148 s, 89.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:58:16,561::task::595::Storage.TaskManager.Task::(_updateState) Task=`b64ca577-813a-47d9-a385-bd210bfb1bfc`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:16,561::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:58:16,562::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182142', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000205184', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:16,562::task::1191::Storage.TaskManager.Task::(prepare) Task=`b64ca577-813a-47d9-a385-bd210bfb1bfc`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182142', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000205184', 'lastCheck': '5.6', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:16,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`b64ca577-813a-47d9-a385-bd210bfb1bfc`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:16,562::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:16,562::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:16,562::task::993::Storage.TaskManager.Task::(_decref) Task=`b64ca577-813a-47d9-a385-bd210bfb1bfc`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:17,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:17,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117441 s, 87.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:19,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:19,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113987 s, 89.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:19,702::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:19,703::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:19,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:19,704::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:19,705::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:19,705::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:19,708::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-21::DEBUG::2014-08-22 09:58:19,709::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:19,709::__init__::225::IOProcess::(_processLogs) (1887) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:19,716::__init__::225::IOProcess::(_processLogs) (1887) Got request for method 'statvfs' >Thread-21::DEBUG::2014-08-22 09:58:19,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00023243 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:19,717::__init__::225::IOProcess::(_processLogs) (1887) Building response >Thread-17::DEBUG::2014-08-22 09:58:19,719::__init__::225::IOProcess::(_processLogs) (1887) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:19,719::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:19,719::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:19,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:19,721::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:19,721::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:19,723::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:19,724::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:19,724::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:19,725::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:19,725::__init__::225::IOProcess::(_processLogs) (1888) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:19,726::__init__::225::IOProcess::(_processLogs) (1888) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:19,726::__init__::225::IOProcess::(_processLogs) (1888) Building response >Thread-17::DEBUG::2014-08-22 09:58:19,726::__init__::225::IOProcess::(_processLogs) (1888) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:19,727::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:19,727::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-47::DEBUG::2014-08-22 09:58:20,964::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:58:20,964::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcc84647-dcf6-40d6-8346-4a689dca471c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:20,964::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:58:20,965::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:20,965::task::1191::Storage.TaskManager.Task::(prepare) Task=`dcc84647-dcf6-40d6-8346-4a689dca471c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:20,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcc84647-dcf6-40d6-8346-4a689dca471c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:20,965::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:20,965::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:20,965::task::993::Storage.TaskManager.Task::(_decref) Task=`dcc84647-dcf6-40d6-8346-4a689dca471c`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 09:58:20,971::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:20,971::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:20,971::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:20,971::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:20,971::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:20,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:20,972::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:20,972::__init__::225::IOProcess::(_processLogs) (1889) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:20,972::__init__::225::IOProcess::(_processLogs) (1889) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:20,972::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1889) Building response >Thread-17::DEBUG::2014-08-22 09:58:20,973::__init__::225::IOProcess::(_processLogs) (1889) Queuing response >Thread-22::DEBUG::2014-08-22 09:58:20,973::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:20,973::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:20,979::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:58:20,980::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000152053 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:20,981::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:20,981::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:20,981::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:20,981::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:20,981::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:20,981::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:20,982::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:20,982::__init__::225::IOProcess::(_processLogs) (1890) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:20,982::__init__::225::IOProcess::(_processLogs) (1890) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:20,982::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1890) Building response >Thread-17::DEBUG::2014-08-22 09:58:20,983::__init__::225::IOProcess::(_processLogs) (1890) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:20,983::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:20,983::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:20,983::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:20,983::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) (1891) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:20,984::__init__::225::IOProcess::(_processLogs) (1891) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) (1891) Building response >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) (1891) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:58:20,985::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) (1892) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) (1892) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) (1892) Building response >Thread-17::DEBUG::2014-08-22 09:58:20,986::__init__::225::IOProcess::(_processLogs) (1892) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:20,987::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:20,987::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:20,987::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:20,987::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:58:20,987::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:20,987::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:20,988::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:20,988::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:20,989::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-46::DEBUG::2014-08-22 09:58:20,989::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-17::DEBUG::2014-08-22 09:58:20,990::__init__::225::IOProcess::(_processLogs) (1893) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:20,990::__init__::225::IOProcess::(_processLogs) (1893) Got request for method 'access' >Thread-46::DEBUG::2014-08-22 09:58:20,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0a97b52-8b47-4aa2-937f-87bd4f416cc2`::moving from state init -> state preparing >Thread-17::DEBUG::2014-08-22 09:58:20,991::__init__::225::IOProcess::(_processLogs) (1893) Building response >Thread-46::INFO::2014-08-22 09:58:20,991::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-17::DEBUG::2014-08-22 09:58:20,991::__init__::225::IOProcess::(_processLogs) (1893) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:20,992::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-46::DEBUG::2014-08-22 09:58:20,993::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0bcc4584-c932-48c1-b49b-f6513dd36d18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-17::DEBUG::2014-08-22 09:58:20,993::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-46::DEBUG::2014-08-22 09:58:20,993::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:58:20,993::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:58:20,993::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0bcc4584-c932-48c1-b49b-f6513dd36d18`::Granted request >Thread-46::DEBUG::2014-08-22 09:58:20,994::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e0a97b52-8b47-4aa2-937f-87bd4f416cc2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:58:20,994::task::993::Storage.TaskManager.Task::(_decref) Task=`e0a97b52-8b47-4aa2-937f-87bd4f416cc2`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:58:20,996::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:20,996::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0a97b52-8b47-4aa2-937f-87bd4f416cc2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:20,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0a97b52-8b47-4aa2-937f-87bd4f416cc2`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:58:20,996::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:58:20,996::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:58:20,996::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:58:20,996::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:58:20,996::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:58:20,996::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:58:20,997::task::993::Storage.TaskManager.Task::(_decref) Task=`e0a97b52-8b47-4aa2-937f-87bd4f416cc2`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:21,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:21,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117321 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:23,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:23,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114711 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:25,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:25,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113966 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:27,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:27,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116415 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:29,724::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:29,725::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:29,725::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:29,725::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:58:29,726::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:29,726::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:29,732::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:58:29,733::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000181159 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:29,734::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) (1894) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) (1894) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) (1894) Building response >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) (1894) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:29,735::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) (1895) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:29,736::__init__::225::IOProcess::(_processLogs) (1895) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:29,737::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|(1895) Building response >Thread-17::DEBUG::2014-08-22 09:58:29,737::__init__::225::IOProcess::(_processLogs) (1895) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:29,738::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:29,738::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:58:29,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:29,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114201 s, 89.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:30,988::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:30,989::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:30,990::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:30,990::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:58:30,991::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:30,991::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) (1896) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) (1896) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) (1896) Building response >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) (1896) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:30,998::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:30,999::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:58:31,012::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.014275 s, 39.9 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:31,012::__init__::225::IOProcess::(_processLogs) (1891) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) (1897) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:31,013::__init__::225::IOProcess::(_processLogs) (1897) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:31,014::__init__::225::IOProcess::(_processLogs) DEBUG|(1897) Building response >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) (1897) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:31,015::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) (1898) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) (1898) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) (1898) Building response >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) (1898) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:31,016::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:31,017::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:31,017::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:58:31,017::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:31,017::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) (1899) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) (1899) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) (1899) Building response >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) (1899) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:31,018::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:31,019::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:58:31,019::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:31,019::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:31,019::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) (1900) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) (1900) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) (1900) Building response >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) (1900) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:31,020::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:58:31,042::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:58:31,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`48e25bb5-937b-433b-8b77-b4bcc50c9187`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:31,043::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:58:31,043::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:31,043::task::1191::Storage.TaskManager.Task::(prepare) Task=`48e25bb5-937b-433b-8b77-b4bcc50c9187`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:31,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`48e25bb5-937b-433b-8b77-b4bcc50c9187`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:31,043::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:31,043::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:31,043::task::993::Storage.TaskManager.Task::(_decref) Task=`48e25bb5-937b-433b-8b77-b4bcc50c9187`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:58:31,085::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:58:31,085::task::595::Storage.TaskManager.Task::(_updateState) Task=`44e778b0-69a4-449e-ae4d-914485fc0bd8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:58:31,086::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:58:31,086::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3f18a2bd-d3b7-48f0-84c5-4401b8ce029f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:58:31,086::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:58:31,086::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:58:31,086::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3f18a2bd-d3b7-48f0-84c5-4401b8ce029f`::Granted request >Thread-46::DEBUG::2014-08-22 09:58:31,087::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`44e778b0-69a4-449e-ae4d-914485fc0bd8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:58:31,087::task::993::Storage.TaskManager.Task::(_decref) Task=`44e778b0-69a4-449e-ae4d-914485fc0bd8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:58:31,089::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:31,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`44e778b0-69a4-449e-ae4d-914485fc0bd8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:31,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`44e778b0-69a4-449e-ae4d-914485fc0bd8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:58:31,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:58:31,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:58:31,089::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:58:31,089::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:58:31,089::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:58:31,089::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:58:31,089::task::993::Storage.TaskManager.Task::(_decref) Task=`44e778b0-69a4-449e-ae4d-914485fc0bd8`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:58:31,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf58fa9c-a271-48cd-b7dd-55656c482916`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:31,782::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:58:31,782::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000181159', 'lastCheck': '2.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.014275', 'lastCheck': '0.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:31,782::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf58fa9c-a271-48cd-b7dd-55656c482916`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000181159', 'lastCheck': '2.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.014275', 'lastCheck': '0.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:31,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf58fa9c-a271-48cd-b7dd-55656c482916`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:31,782::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:31,782::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:31,782::task::993::Storage.TaskManager.Task::(_decref) Task=`cf58fa9c-a271-48cd-b7dd-55656c482916`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:31,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:32,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212273 s, 48.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:34,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:34,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123098 s, 83.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:36,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:36,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116512 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:38,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:38,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112679 s, 90.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:39,738::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:39,738::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:39,738::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:39,738::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:39,738::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:39,738::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:39,739::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:39,739::__init__::225::IOProcess::(_processLogs) (1901) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:39,739::__init__::225::IOProcess::(_processLogs) (1901) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:39,739::__init__::225::IOProcess::(_processLogs) (1901) Building response >Thread-17::DEBUG::2014-08-22 09:58:39,739::__init__::225::IOProcess::(_processLogs) (1901) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:39,739::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:58:39,740::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:58:39,740::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 09:58:39,747::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000270992 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:39,748::__init__::225::IOProcess::(_processLogs) (1902) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:39,749::__init__::225::IOProcess::(_processLogs) (1902) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:39,749::__init__::225::IOProcess::(_processLogs) DEBUG|(1902) Building response >Thread-17::DEBUG::2014-08-22 09:58:39,750::__init__::225::IOProcess::(_processLogs) (1902) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:39,750::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:39,750::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:58:40,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:40,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116119 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:41,020::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:41,020::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:41,020::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:41,021::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:58:41,021::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:41,021::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:41,028::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:58:41,029::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000217416 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:41,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) (1903) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) (1903) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) (1903) Building response >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) (1903) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:41,030::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) (1904) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:41,031::__init__::225::IOProcess::(_processLogs) (1904) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:41,032::__init__::225::IOProcess::(_processLogs) DEBUG|(1904) Building response >Thread-17::DEBUG::2014-08-22 09:58:41,032::__init__::225::IOProcess::(_processLogs) (1904) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:41,032::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:41,032::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:41,033::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:41,033::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:58:41,033::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:41,033::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:41,033::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) (1905) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) (1905) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) (1905) Building response >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) (1905) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:58:41,034::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:41,035::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:41,035::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:41,035::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:41,035::__init__::225::IOProcess::(_processLogs) (1906) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:41,035::__init__::225::IOProcess::(_processLogs) (1906) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:41,035::__init__::225::IOProcess::(_processLogs) (1906) Building response >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) (1906) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:41,036::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:41,037::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:41,037::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:41,037::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:41,037::__init__::225::IOProcess::(_processLogs) (1907) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:41,037::__init__::225::IOProcess::(_processLogs) (1907) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:41,038::__init__::225::IOProcess::(_processLogs) (1907) Building response >Thread-17::DEBUG::2014-08-22 09:58:41,038::__init__::225::IOProcess::(_processLogs) (1907) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:41,038::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:41,038::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:58:41,250::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:58:41,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`76b8d166-6d42-4b6f-8c1d-462f837d9fef`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:41,251::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:58:41,251::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:41,251::task::1191::Storage.TaskManager.Task::(prepare) Task=`76b8d166-6d42-4b6f-8c1d-462f837d9fef`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:41,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`76b8d166-6d42-4b6f-8c1d-462f837d9fef`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:41,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:41,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:41,251::task::993::Storage.TaskManager.Task::(_decref) Task=`76b8d166-6d42-4b6f-8c1d-462f837d9fef`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:58:41,289::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:58:41,289::task::595::Storage.TaskManager.Task::(_updateState) Task=`2addd069-1173-4c10-af8f-08b1fff8ba7a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:58:41,289::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:58:41,289::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2993f8d0-ca50-42e1-a08f-6e342606383a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:58:41,289::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:58:41,289::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:58:41,290::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2993f8d0-ca50-42e1-a08f-6e342606383a`::Granted request >Thread-46::DEBUG::2014-08-22 09:58:41,290::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2addd069-1173-4c10-af8f-08b1fff8ba7a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:58:41,290::task::993::Storage.TaskManager.Task::(_decref) Task=`2addd069-1173-4c10-af8f-08b1fff8ba7a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:58:41,292::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:41,292::task::1191::Storage.TaskManager.Task::(prepare) Task=`2addd069-1173-4c10-af8f-08b1fff8ba7a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:41,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`2addd069-1173-4c10-af8f-08b1fff8ba7a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:58:41,292::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:58:41,292::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:58:41,292::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:58:41,292::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:58:41,292::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:58:41,292::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:58:41,293::task::993::Storage.TaskManager.Task::(_decref) Task=`2addd069-1173-4c10-af8f-08b1fff8ba7a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:42,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:42,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118606 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:44,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:44,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114311 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:46,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:46,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118212 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:58:46,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`2673b4f0-717d-4cab-b6c3-c56f0b23d4d0`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:46,997::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:58:46,997::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000270992', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000217416', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:46,997::task::1191::Storage.TaskManager.Task::(prepare) Task=`2673b4f0-717d-4cab-b6c3-c56f0b23d4d0`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000270992', 'lastCheck': '7.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000217416', 'lastCheck': '6.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:58:46,997::task::595::Storage.TaskManager.Task::(_updateState) Task=`2673b4f0-717d-4cab-b6c3-c56f0b23d4d0`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:46,997::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:46,997::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:46,997::task::993::Storage.TaskManager.Task::(_decref) Task=`2673b4f0-717d-4cab-b6c3-c56f0b23d4d0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:48,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:48,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115163 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:49,750::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) (1908) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:49,751::__init__::225::IOProcess::(_processLogs) (1908) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:49,752::__init__::225::IOProcess::(_processLogs) DEBUG|(1908) Building response >Thread-21::DEBUG::2014-08-22 09:58:49,753::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:49,753::__init__::225::IOProcess::(_processLogs) (1908) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:49,759::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:58:49,760::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000196181 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:49,760::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:49,761::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) (1909) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:49,762::__init__::225::IOProcess::(_processLogs) (1909) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:49,763::__init__::225::IOProcess::(_processLogs) DEBUG|(1909) Building response >Thread-17::DEBUG::2014-08-22 09:58:49,764::__init__::225::IOProcess::(_processLogs) (1909) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:49,764::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:49,764::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:58:50,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:50,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117383 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:51,038::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:51,038::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:51,038::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:51,038::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 09:58:51,039::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:51,039::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:51,046::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 09:58:51,047::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000167966 s, 3.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:51,047::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) (1910) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) (1910) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) (1910) Building response >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) (1910) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:51,048::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) (1911) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) (1911) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) (1911) Building response >Thread-17::DEBUG::2014-08-22 09:58:51,049::__init__::225::IOProcess::(_processLogs) (1911) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:51,050::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:58:51,050::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:51,051::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:51,051::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:58:51,051::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:51,051::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) (1912) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) (1912) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) (1912) Building response >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) (1912) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:51,052::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:51,053::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:58:51,053::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:51,053::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:51,053::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:51,053::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:51,053::__init__::225::IOProcess::(_processLogs) (1913) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) (1913) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) (1913) Building response >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) (1913) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:58:51,054::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:51,055::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:51,055::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:51,055::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:51,055::__init__::225::IOProcess::(_processLogs) (1914) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:58:51,055::__init__::225::IOProcess::(_processLogs) (1914) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:58:51,056::__init__::225::IOProcess::(_processLogs) (1914) Building response >Thread-17::DEBUG::2014-08-22 09:58:51,056::__init__::225::IOProcess::(_processLogs) (1914) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:51,056::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:58:51,056::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:58:51,344::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:58:51,345::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5a0107e-80b4-4b8c-bb46-0fc95b4484fa`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:58:51,345::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:58:51,345::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:51,345::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5a0107e-80b4-4b8c-bb46-0fc95b4484fa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:58:51,345::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5a0107e-80b4-4b8c-bb46-0fc95b4484fa`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:58:51,345::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:58:51,345::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:58:51,345::task::993::Storage.TaskManager.Task::(_decref) Task=`b5a0107e-80b4-4b8c-bb46-0fc95b4484fa`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:58:51,384::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:58:51,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`00c7c8ee-af58-4cd3-9b06-23d84fab4d29`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:58:51,385::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:58:51,385::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d5571b64-fbbe-403e-997c-5852d12b7ddf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:58:51,385::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:58:51,385::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:58:51,385::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d5571b64-fbbe-403e-997c-5852d12b7ddf`::Granted request >Thread-46::DEBUG::2014-08-22 09:58:51,386::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`00c7c8ee-af58-4cd3-9b06-23d84fab4d29`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:58:51,386::task::993::Storage.TaskManager.Task::(_decref) Task=`00c7c8ee-af58-4cd3-9b06-23d84fab4d29`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:58:51,388::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:51,388::task::1191::Storage.TaskManager.Task::(prepare) Task=`00c7c8ee-af58-4cd3-9b06-23d84fab4d29`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:58:51,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`00c7c8ee-af58-4cd3-9b06-23d84fab4d29`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:58:51,389::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:58:51,389::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:58:51,389::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:58:51,389::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:58:51,389::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:58:51,389::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:58:51,389::task::993::Storage.TaskManager.Task::(_decref) Task=`00c7c8ee-af58-4cd3-9b06-23d84fab4d29`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:58:52,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:52,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117979 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:54,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:54,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115515 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:56,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:56,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115578 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:58:58,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:58:58,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123229 s, 83.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:59,764::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:59,764::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:59,764::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:59,764::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:59,765::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:59,765::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:59,765::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:59,765::__init__::225::IOProcess::(_processLogs) (1915) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:59,765::__init__::225::IOProcess::(_processLogs) (1915) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:59,765::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1915) Building response >Thread-21::DEBUG::2014-08-22 09:58:59,766::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:58:59,766::__init__::225::IOProcess::(_processLogs) (1915) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:59,773::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:58:59,774::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178933 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:58:59,774::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:58:59,775::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:58:59,775::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:58:59,775::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:58:59,775::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:58:59,775::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:58:59,775::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:58:59,776::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:58:59,776::__init__::225::IOProcess::(_processLogs) (1916) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:58:59,776::__init__::225::IOProcess::(_processLogs) (1916) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:58:59,776::__init__::225::IOProcess::(_processLogs) (1916) Building response >Thread-17::DEBUG::2014-08-22 09:58:59,776::__init__::225::IOProcess::(_processLogs) (1916) Queuing response >Thread-17::DEBUG::2014-08-22 09:58:59,776::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:58:59,777::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:59:00,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:00,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012018 s, 85.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:01,056::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:01,056::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:01,056::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:01,056::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:01,056::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:01,056::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:01,057::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:01,057::__init__::225::IOProcess::(_processLogs) (1917) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:01,057::__init__::225::IOProcess::(_processLogs) (1917) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:01,057::__init__::225::IOProcess::(_processLogs) (1917) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:01,058::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:59:01,058::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:01,058::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:59:01,065::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0002206 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:01,065::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:01,065::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:01,066::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:01,066::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:01,066::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:01,066::__init__::225::IOProcess::(_processLogs) (1918) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:01,066::__init__::225::IOProcess::(_processLogs) (1918) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:01,067::__init__::225::IOProcess::(_processLogs) (1918) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:01,067::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:01,067::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:01,067::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:59:01,068::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:01,068::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:01,068::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:01,068::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:01,068::__init__::225::IOProcess::(_processLogs) (1919) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:01,068::__init__::225::IOProcess::(_processLogs) (1919) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) (1919) Building response >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) (1919) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:01,069::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:01,070::__init__::225::IOProcess::(_processLogs) (1920) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:01,070::__init__::225::IOProcess::(_processLogs) (1920) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:01,070::__init__::225::IOProcess::(_processLogs) (1920) Building response >Thread-17::DEBUG::2014-08-22 09:59:01,070::__init__::225::IOProcess::(_processLogs) (1920) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:01,070::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:01,070::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:59:01,071::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:01,071::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:01,071::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) (1921) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) (1921) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) (1921) Building response >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) (1921) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:01,072::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:59:01,438::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:59:01,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`471d833d-e230-4270-9db6-9063986a279c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:01,439::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:59:01,441::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:01,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`471d833d-e230-4270-9db6-9063986a279c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:01,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`471d833d-e230-4270-9db6-9063986a279c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:01,442::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:01,442::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:01,443::task::993::Storage.TaskManager.Task::(_decref) Task=`471d833d-e230-4270-9db6-9063986a279c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:59:01,519::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:59:01,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8a7140c-bf46-4b6a-9c59-9f0b457378df`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:59:01,520::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:59:01,520::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a86d1b8f-9bb1-4072-b90c-f3e6dad27b01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:59:01,520::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:59:01,520::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:59:01,520::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a86d1b8f-9bb1-4072-b90c-f3e6dad27b01`::Granted request >Thread-46::DEBUG::2014-08-22 09:59:01,521::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8a7140c-bf46-4b6a-9c59-9f0b457378df`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:59:01,521::task::993::Storage.TaskManager.Task::(_decref) Task=`a8a7140c-bf46-4b6a-9c59-9f0b457378df`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:59:01,523::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:01,523::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8a7140c-bf46-4b6a-9c59-9f0b457378df`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:01,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8a7140c-bf46-4b6a-9c59-9f0b457378df`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:59:01,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:59:01,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:59:01,523::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:59:01,523::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:59:01,523::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:59:01,524::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:59:01,524::task::993::Storage.TaskManager.Task::(_decref) Task=`a8a7140c-bf46-4b6a-9c59-9f0b457378df`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:59:02,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2042181-f07a-4fd0-9771-f1bd2785c64d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:02,178::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:59:02,179::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178933', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002206', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:02,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2042181-f07a-4fd0-9771-f1bd2785c64d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000178933', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002206', 'lastCheck': '1.1', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:02,181::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2042181-f07a-4fd0-9771-f1bd2785c64d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:02,181::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:02,182::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:02,183::task::993::Storage.TaskManager.Task::(_decref) Task=`f2042181-f07a-4fd0-9771-f1bd2785c64d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:02,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:02,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188005 s, 54.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:04,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:05,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113962 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:07,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:07,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116147 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:09,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:09,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114149 s, 89.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) (1922) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:09,778::__init__::225::IOProcess::(_processLogs) (1922) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:09,779::__init__::225::IOProcess::(_processLogs) (1922) Queuing response >Thread-21::DEBUG::2014-08-22 09:59:09,780::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:09,780::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:09,786::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:59:09,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000167858 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) (1923) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:09,788::__init__::225::IOProcess::(_processLogs) (1923) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:09,789::__init__::225::IOProcess::(_processLogs) (1923) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:09,790::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:09,790::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:11,072::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:11,072::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:11,072::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:11,072::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:11,072::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:11,073::__init__::225::IOProcess::(_processLogs) (1924) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:11,073::__init__::225::IOProcess::(_processLogs) (1924) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:11,073::__init__::225::IOProcess::(_processLogs) (1924) Building response >Thread-17::DEBUG::2014-08-22 09:59:11,073::__init__::225::IOProcess::(_processLogs) (1924) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:11,073::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:59:11,073::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:59:11,080::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000259641 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:11,080::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:11,080::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:11,080::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:11,081::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:11,081::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:11,081::__init__::225::IOProcess::(_processLogs) (1925) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:11,081::__init__::225::IOProcess::(_processLogs) (1925) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:11,081::__init__::225::IOProcess::(_processLogs) (1925) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:11,082::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:11,082::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:11,082::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:59:11,082::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:11,083::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:11,083::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:11,083::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:11,083::__init__::225::IOProcess::(_processLogs) (1926) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:11,083::__init__::225::IOProcess::(_processLogs) (1926) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:11,083::__init__::225::IOProcess::(_processLogs) (1926) Building response >Thread-17::DEBUG::2014-08-22 09:59:11,084::__init__::225::IOProcess::(_processLogs) (1926) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:11,084::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:11,084::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:11,084::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:59:11,084::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:11,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) (1927) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) (1927) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) (1927) Building response >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) (1927) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:11,085::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:59:11,086::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:11,086::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:11,086::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) (1928) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) (1928) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) (1928) Building response >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) (1928) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:11,087::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:59:11,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:11,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117008 s, 87.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:59:11,573::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:59:11,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`30ab0bc4-ff40-49c2-9293-769cdb4b939b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:11,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:59:11,574::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:11,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`30ab0bc4-ff40-49c2-9293-769cdb4b939b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:11,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`30ab0bc4-ff40-49c2-9293-769cdb4b939b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:11,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:11,575::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:11,575::task::993::Storage.TaskManager.Task::(_decref) Task=`30ab0bc4-ff40-49c2-9293-769cdb4b939b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:59:11,600::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:59:11,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`28ae2de8-2663-4e48-bf40-6973c424c074`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:59:11,601::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:59:11,601::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8ecfa524-24db-4573-9ec8-95683e0026bf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:59:11,601::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:59:11,601::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:59:11,601::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`8ecfa524-24db-4573-9ec8-95683e0026bf`::Granted request >Thread-46::DEBUG::2014-08-22 09:59:11,601::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`28ae2de8-2663-4e48-bf40-6973c424c074`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:59:11,601::task::993::Storage.TaskManager.Task::(_decref) Task=`28ae2de8-2663-4e48-bf40-6973c424c074`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:59:11,603::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:11,604::task::1191::Storage.TaskManager.Task::(prepare) Task=`28ae2de8-2663-4e48-bf40-6973c424c074`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:11,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`28ae2de8-2663-4e48-bf40-6973c424c074`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:59:11,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:59:11,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:59:11,604::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:59:11,604::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:59:11,604::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:59:11,604::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:59:11,604::task::993::Storage.TaskManager.Task::(_decref) Task=`28ae2de8-2663-4e48-bf40-6973c424c074`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:13,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:13,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113486 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:15,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:15,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161564 s, 63.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:17,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:17,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115054 s, 89.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:59:17,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ce2b26f-e0b4-40f0-8945-569c6b64c04f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:17,456::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:59:17,456::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000167858', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000259641', 'lastCheck': '6.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:17,456::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ce2b26f-e0b4-40f0-8945-569c6b64c04f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000167858', 'lastCheck': '7.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000259641', 'lastCheck': '6.4', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:17,457::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ce2b26f-e0b4-40f0-8945-569c6b64c04f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:17,457::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:17,457::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:17,457::task::993::Storage.TaskManager.Task::(_decref) Task=`9ce2b26f-e0b4-40f0-8945-569c6b64c04f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:19,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:19,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117699 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:19,790::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:19,790::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:19,791::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:19,791::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:19,791::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:19,791::__init__::225::IOProcess::(_processLogs) (1929) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:19,791::__init__::225::IOProcess::(_processLogs) (1929) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:19,791::__init__::225::IOProcess::(_processLogs) (1929) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:19,792::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:59:19,792::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:19,793::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:59:19,799::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00024383 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:19,800::__init__::225::IOProcess::(_processLogs) (1930) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:19,801::__init__::225::IOProcess::(_processLogs) (1930) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:19,801::__init__::225::IOProcess::(_processLogs) (1930) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:19,802::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:19,802::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) (1931) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:21,087::__init__::225::IOProcess::(_processLogs) (1931) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:21,088::__init__::225::IOProcess::(_processLogs) (1931) Queuing response >Thread-22::DEBUG::2014-08-22 09:59:21,089::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:21,089::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:21,095::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:59:21,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000409445 s, 1.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:21,096::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:21,096::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:21,097::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:21,097::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:21,097::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:21,097::__init__::225::IOProcess::(_processLogs) (1932) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:21,097::__init__::225::IOProcess::(_processLogs) (1932) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:21,098::__init__::225::IOProcess::(_processLogs) (1932) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:21,098::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:21,098::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:21,098::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:59:21,099::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:21,099::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:21,099::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:21,099::__init__::225::IOProcess::(_processLogs) (1933) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:21,099::__init__::225::IOProcess::(_processLogs) (1933) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:21,099::__init__::225::IOProcess::(_processLogs) (1933) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:21,100::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:21,100::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:21,100::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:59:21,100::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:21,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) (1934) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) (1934) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) (1934) Building response >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) (1934) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:21,101::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:59:21,102::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:21,102::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:21,102::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:21,102::__init__::225::IOProcess::(_processLogs) (1935) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:21,103::__init__::225::IOProcess::(_processLogs) (1935) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:21,103::__init__::225::IOProcess::(_processLogs) (1935) Building response >Thread-17::DEBUG::2014-08-22 09:59:21,103::__init__::225::IOProcess::(_processLogs) (1935) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:21,103::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:21,103::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:59:21,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:21,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120124 s, 85.2 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:59:21,652::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:59:21,652::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f72abe5-8016-4a37-a235-f355764bd46c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:21,652::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:59:21,652::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:21,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f72abe5-8016-4a37-a235-f355764bd46c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:21,653::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f72abe5-8016-4a37-a235-f355764bd46c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:21,653::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:21,653::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:21,653::task::993::Storage.TaskManager.Task::(_decref) Task=`1f72abe5-8016-4a37-a235-f355764bd46c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:59:21,687::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:59:21,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b00e500-a7d7-4362-8ad7-607ee0a85764`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:59:21,687::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:59:21,688::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ca925de4-8f51-4518-930f-73170f952729`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:59:21,688::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:59:21,688::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:59:21,688::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ca925de4-8f51-4518-930f-73170f952729`::Granted request >Thread-46::DEBUG::2014-08-22 09:59:21,688::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1b00e500-a7d7-4362-8ad7-607ee0a85764`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:59:21,688::task::993::Storage.TaskManager.Task::(_decref) Task=`1b00e500-a7d7-4362-8ad7-607ee0a85764`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:59:21,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:21,690::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b00e500-a7d7-4362-8ad7-607ee0a85764`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:21,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b00e500-a7d7-4362-8ad7-607ee0a85764`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:59:21,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:59:21,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:59:21,691::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:59:21,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:59:21,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:59:21,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:59:21,691::task::993::Storage.TaskManager.Task::(_decref) Task=`1b00e500-a7d7-4362-8ad7-607ee0a85764`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:23,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:23,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114121 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:25,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:25,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116789 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:27,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:27,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116468 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:29,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:29,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113984 s, 89.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) (1936) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:29,803::__init__::225::IOProcess::(_processLogs) (1936) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:29,804::__init__::225::IOProcess::(_processLogs) (1936) Queuing response >Thread-21::DEBUG::2014-08-22 09:59:29,805::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:29,805::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:29,811::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 09:59:29,812::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000268073 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:29,812::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:29,813::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:29,813::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:29,813::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:29,813::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:29,813::__init__::225::IOProcess::(_processLogs) (1937) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:29,813::__init__::225::IOProcess::(_processLogs) (1937) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:29,814::__init__::225::IOProcess::(_processLogs) (1937) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:29,815::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:29,815::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) (1938) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) (1938) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:31,103::__init__::225::IOProcess::(_processLogs) (1938) Building response >Thread-17::DEBUG::2014-08-22 09:59:31,104::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:31,104::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:59:31,105::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:59:31,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000257407 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) (1939) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:31,112::__init__::225::IOProcess::(_processLogs) (1939) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:31,113::__init__::225::IOProcess::(_processLogs) DEBUG|(1939) Building response >Thread-17::DEBUG::2014-08-22 09:59:31,114::__init__::225::IOProcess::(_processLogs) (1939) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:31,114::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:31,114::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:31,114::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:31,114::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:59:31,114::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) (1940) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) (1940) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) (1940) Building response >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) (1940) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:31,115::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:31,116::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:31,116::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:31,116::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:59:31,116::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:31,116::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) (1941) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) (1941) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) (1941) Building response >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) (1941) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:31,117::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:31,118::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:59:31,118::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:31,118::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) (1942) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) (1942) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) (1942) Building response >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) (1942) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:31,119::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 09:59:31,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:31,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116206 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:59:31,741::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [739d4759] >Thread-47::DEBUG::2014-08-22 09:59:31,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`296c1cbf-eb93-4858-b821-096245c4387c`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:31,742::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:59:31,742::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:31,742::task::1191::Storage.TaskManager.Task::(prepare) Task=`296c1cbf-eb93-4858-b821-096245c4387c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:31,742::task::595::Storage.TaskManager.Task::(_updateState) Task=`296c1cbf-eb93-4858-b821-096245c4387c`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:31,742::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:31,742::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:31,742::task::993::Storage.TaskManager.Task::(_decref) Task=`296c1cbf-eb93-4858-b821-096245c4387c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:59:31,775::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [739d4759] >Thread-46::DEBUG::2014-08-22 09:59:31,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f8712af-f4d8-45f6-be5f-03e72dde848a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:59:31,775::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:59:31,776::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`203c649b-c662-4cec-b34b-2a67460efc09`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:59:31,776::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:59:31,776::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:59:31,776::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`203c649b-c662-4cec-b34b-2a67460efc09`::Granted request >Thread-46::DEBUG::2014-08-22 09:59:31,776::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3f8712af-f4d8-45f6-be5f-03e72dde848a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:59:31,776::task::993::Storage.TaskManager.Task::(_decref) Task=`3f8712af-f4d8-45f6-be5f-03e72dde848a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:59:31,778::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:31,778::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f8712af-f4d8-45f6-be5f-03e72dde848a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:31,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f8712af-f4d8-45f6-be5f-03e72dde848a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:59:31,779::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:59:31,779::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:59:31,779::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:59:31,779::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:59:31,779::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:59:31,779::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:59:31,779::task::993::Storage.TaskManager.Task::(_decref) Task=`3f8712af-f4d8-45f6-be5f-03e72dde848a`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 09:59:32,645::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d26352b-82c8-4d12-9b1b-6f8d3eb0941f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:32,646::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:59:32,647::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000268073', 'lastCheck': '2.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000257407', 'lastCheck': '1.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:32,648::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d26352b-82c8-4d12-9b1b-6f8d3eb0941f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000268073', 'lastCheck': '2.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000257407', 'lastCheck': '1.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:32,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d26352b-82c8-4d12-9b1b-6f8d3eb0941f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:32,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:32,650::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:32,650::task::993::Storage.TaskManager.Task::(_decref) Task=`6d26352b-82c8-4d12-9b1b-6f8d3eb0941f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:33,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:33,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115212 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:35,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:35,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117692 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:37,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:37,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115359 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:39,815::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:39,815::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:39,815::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:39,816::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:59:39,816::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:39,817::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:39,823::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:59:39,824::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000229188 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:39,824::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:39,825::__init__::225::IOProcess::(_processLogs) (1943) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:39,825::__init__::225::IOProcess::(_processLogs) (1943) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:39,825::__init__::225::IOProcess::(_processLogs) (1943) Building response >Thread-17::DEBUG::2014-08-22 09:59:39,825::__init__::225::IOProcess::(_processLogs) (1943) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:39,825::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) (1944) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:39,826::__init__::225::IOProcess::(_processLogs) (1944) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:39,827::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1944) Building response >Thread-17::DEBUG::2014-08-22 09:59:39,828::__init__::225::IOProcess::(_processLogs) (1944) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:39,828::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:39,828::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:59:39,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:39,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113127 s, 90.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:41,119::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:41,119::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:41,119::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:41,119::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) (1945) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) (1945) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) (1945) Building response >Thread-17::DEBUG::2014-08-22 09:59:41,120::__init__::225::IOProcess::(_processLogs) (1945) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:41,121::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 09:59:41,121::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 09:59:41,121::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 09:59:41,128::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000262599 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:41,129::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:41,129::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:41,129::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) (1946) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) (1946) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:41,130::__init__::225::IOProcess::(_processLogs) (1946) Building response >Thread-17::DEBUG::2014-08-22 09:59:41,131::__init__::225::IOProcess::(_processLogs) (1946) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:41,131::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:41,131::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:41,131::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:41,131::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:59:41,131::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) (1947) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) (1947) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) (1947) Building response >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) (1947) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:41,132::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:41,133::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:41,133::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:59:41,133::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:41,133::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:41,133::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) (1948) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) (1948) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) (1948) Building response >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) (1948) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:41,134::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:59:41,135::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:41,135::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:41,135::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) (1949) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) (1949) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) (1949) Building response >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) (1949) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:41,136::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:59:41,828::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:59:41,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`4541a835-92e0-4339-8ca1-99ba59c74cab`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:41,829::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:59:41,829::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:41,829::task::1191::Storage.TaskManager.Task::(prepare) Task=`4541a835-92e0-4339-8ca1-99ba59c74cab`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:41,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`4541a835-92e0-4339-8ca1-99ba59c74cab`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:41,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:41,829::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:41,829::task::993::Storage.TaskManager.Task::(_decref) Task=`4541a835-92e0-4339-8ca1-99ba59c74cab`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:59:41,862::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:59:41,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd5abc9b-71dd-4836-84d6-ecf5adf91016`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:59:41,862::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:59:41,863::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0df24837-ebe5-4052-8976-a6f03dad2065`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:59:41,863::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:59:41,863::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:59:41,863::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0df24837-ebe5-4052-8976-a6f03dad2065`::Granted request >Thread-46::DEBUG::2014-08-22 09:59:41,863::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fd5abc9b-71dd-4836-84d6-ecf5adf91016`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:59:41,863::task::993::Storage.TaskManager.Task::(_decref) Task=`fd5abc9b-71dd-4836-84d6-ecf5adf91016`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:59:41,865::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:41,865::task::1191::Storage.TaskManager.Task::(prepare) Task=`fd5abc9b-71dd-4836-84d6-ecf5adf91016`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:41,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd5abc9b-71dd-4836-84d6-ecf5adf91016`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:59:41,865::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:59:41,865::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:59:41,866::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:59:41,866::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:59:41,866::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:59:41,866::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:59:41,866::task::993::Storage.TaskManager.Task::(_decref) Task=`fd5abc9b-71dd-4836-84d6-ecf5adf91016`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:41,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:41,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118058 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:43,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:43,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115552 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:45,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:46,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116826 s, 87.7 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 09:59:47,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`608b09a0-0bf9-4f90-a390-103f9fd05b6f`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:47,952::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 09:59:47,952::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000229188', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000262599', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:47,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`608b09a0-0bf9-4f90-a390-103f9fd05b6f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000229188', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000262599', 'lastCheck': '6.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 09:59:47,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`608b09a0-0bf9-4f90-a390-103f9fd05b6f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:47,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:47,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:47,953::task::993::Storage.TaskManager.Task::(_decref) Task=`608b09a0-0bf9-4f90-a390-103f9fd05b6f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:48,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:48,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116109 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:49,828::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:49,828::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) (1950) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:49,829::__init__::225::IOProcess::(_processLogs) (1950) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:49,830::__init__::225::IOProcess::(_processLogs) DEBUG|(1950) Building response >Thread-21::DEBUG::2014-08-22 09:59:49,830::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:49,831::__init__::225::IOProcess::(_processLogs) (1950) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:49,837::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 09:59:49,838::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000185679 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:49,838::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:49,839::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) (1951) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:49,840::__init__::225::IOProcess::(_processLogs) (1951) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:49,841::__init__::225::IOProcess::(_processLogs) DEBUG|(1951) Building response >Thread-17::DEBUG::2014-08-22 09:59:49,842::__init__::225::IOProcess::(_processLogs) (1951) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:49,842::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:49,842::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 09:59:50,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:50,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116986 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:51,136::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:51,136::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:51,136::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:51,136::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:51,136::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:51,137::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:51,137::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:51,137::__init__::225::IOProcess::(_processLogs) (1952) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:51,137::__init__::225::IOProcess::(_processLogs) (1952) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:51,137::__init__::225::IOProcess::(_processLogs) DEBUG|(1952) Building response >Thread-22::DEBUG::2014-08-22 09:59:51,138::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:51,138::__init__::225::IOProcess::(_processLogs) (1952) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:51,145::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 09:59:51,145::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000186787 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:51,146::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:51,147::__init__::225::IOProcess::(_processLogs) (1953) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:51,148::__init__::225::IOProcess::(_processLogs) (1953) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:51,148::__init__::225::IOProcess::(_processLogs) DEBUG|(1953) Building response >Thread-17::DEBUG::2014-08-22 09:59:51,149::__init__::225::IOProcess::(_processLogs) (1953) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:51,149::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:51,149::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:51,149::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:51,149::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 09:59:51,149::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) (1954) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) (1954) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:51,150::__init__::225::IOProcess::(_processLogs) (1954) Building response >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) (1954) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:51,151::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) (1955) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) (1955) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) (1955) Building response >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) (1955) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:51,152::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 09:59:51,153::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:51,153::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 09:59:51,153::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:51,153::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:51,153::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) (1956) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) (1956) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) (1956) Building response >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) (1956) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:51,154::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 09:59:51,923::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 09:59:51,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`716b99b6-d083-4f77-a61e-09d0909d4794`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 09:59:51,924::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 09:59:51,924::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:51,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`716b99b6-d083-4f77-a61e-09d0909d4794`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 09:59:51,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`716b99b6-d083-4f77-a61e-09d0909d4794`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 09:59:51,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 09:59:51,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 09:59:51,924::task::993::Storage.TaskManager.Task::(_decref) Task=`716b99b6-d083-4f77-a61e-09d0909d4794`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 09:59:51,963::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 09:59:51,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e7a1405-7239-428f-b9b2-e4542c555123`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 09:59:51,964::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 09:59:51,964::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`558b1336-e55d-4d7d-bbc7-39b5c9f849fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 09:59:51,964::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 09:59:51,964::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 09:59:51,964::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`558b1336-e55d-4d7d-bbc7-39b5c9f849fc`::Granted request >Thread-46::DEBUG::2014-08-22 09:59:51,964::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6e7a1405-7239-428f-b9b2-e4542c555123`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 09:59:51,964::task::993::Storage.TaskManager.Task::(_decref) Task=`6e7a1405-7239-428f-b9b2-e4542c555123`::ref 1 aborting False >Thread-46::INFO::2014-08-22 09:59:51,966::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:51,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e7a1405-7239-428f-b9b2-e4542c555123`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 09:59:51,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e7a1405-7239-428f-b9b2-e4542c555123`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 09:59:51,967::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 09:59:51,967::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 09:59:51,967::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 09:59:51,967::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 09:59:51,967::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 09:59:51,967::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 09:59:51,967::task::993::Storage.TaskManager.Task::(_decref) Task=`6e7a1405-7239-428f-b9b2-e4542c555123`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 09:59:52,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:52,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119562 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:54,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:54,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117804 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:56,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:56,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115152 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 09:59:58,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 09:59:58,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117634 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:59,842::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:59,842::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:59,842::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:59,843::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 09:59:59,844::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 09:59:59,844::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:59,850::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 09:59:59,851::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000216844 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 09:59:59,851::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:59,852::__init__::225::IOProcess::(_processLogs) (1957) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:59,852::__init__::225::IOProcess::(_processLogs) (1957) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) (1957) Building response >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) (1957) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 09:59:59,853::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 09:59:59,854::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 09:59:59,854::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 09:59:59,854::__init__::225::IOProcess::(_processLogs) (1958) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 09:59:59,854::__init__::225::IOProcess::(_processLogs) (1958) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 09:59:59,854::__init__::225::IOProcess::(_processLogs) DEBUG|(1958) Building response >Thread-17::DEBUG::2014-08-22 09:59:59,855::__init__::225::IOProcess::(_processLogs) (1958) Queuing response >Thread-17::DEBUG::2014-08-22 09:59:59,855::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 09:59:59,855::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:00:00,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:00,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140409 s, 72.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:01,154::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:01,154::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:01,155::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:01,155::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-22::DEBUG::2014-08-22 10:00:01,156::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:01,156::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:01,162::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:00:01,163::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000159972 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:01,163::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) (1959) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) (1959) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) (1959) Building response >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) (1959) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:01,164::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) (1960) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) (1960) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) (1960) Building response >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) (1960) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:01,165::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:01,166::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:01,167::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:01,167::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:00:01,167::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:01,167::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:01,167::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:01,167::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) (1961) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) (1961) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) (1961) Building response >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) (1961) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:00:01,168::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) (1962) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) (1962) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:01,169::__init__::225::IOProcess::(_processLogs) (1962) Building response >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) (1962) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:01,170::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) (1963) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) (1963) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) (1963) Building response >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) (1963) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:01,171::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:01,172::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 10:00:02,073::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:00:02,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`299a555d-729b-484a-89d9-be8764582fa9`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:02,076::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:00:02,077::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:02,081::task::1191::Storage.TaskManager.Task::(prepare) Task=`299a555d-729b-484a-89d9-be8764582fa9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:02,082::task::595::Storage.TaskManager.Task::(_updateState) Task=`299a555d-729b-484a-89d9-be8764582fa9`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:02,082::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:02,083::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:02,083::task::993::Storage.TaskManager.Task::(_decref) Task=`299a555d-729b-484a-89d9-be8764582fa9`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:00:02,112::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:00:02,114::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb8939bb-15df-4619-94f8-f84289af069a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:00:02,114::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:00:02,116::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`aa33ddb2-e897-461e-8e89-3d381cb0b174`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:00:02,117::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:00:02,118::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:00:02,118::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`aa33ddb2-e897-461e-8e89-3d381cb0b174`::Granted request >Thread-46::DEBUG::2014-08-22 10:00:02,120::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fb8939bb-15df-4619-94f8-f84289af069a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:00:02,121::task::993::Storage.TaskManager.Task::(_decref) Task=`fb8939bb-15df-4619-94f8-f84289af069a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:00:02,135::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:02,136::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb8939bb-15df-4619-94f8-f84289af069a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:02,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb8939bb-15df-4619-94f8-f84289af069a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:00:02,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:00:02,138::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:00:02,139::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:00:02,139::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:00:02,140::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:00:02,140::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:00:02,142::task::993::Storage.TaskManager.Task::(_decref) Task=`fb8939bb-15df-4619-94f8-f84289af069a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:02,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:02,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172033 s, 59.5 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:00:03,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`5296f30e-aedf-42c6-870f-1d8828f65d73`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:03,139::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:00:03,139::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000216844', 'lastCheck': '3.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000159972', 'lastCheck': '2.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:03,139::task::1191::Storage.TaskManager.Task::(prepare) Task=`5296f30e-aedf-42c6-870f-1d8828f65d73`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000216844', 'lastCheck': '3.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000159972', 'lastCheck': '2.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:03,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`5296f30e-aedf-42c6-870f-1d8828f65d73`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:03,139::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:03,140::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:03,140::task::993::Storage.TaskManager.Task::(_decref) Task=`5296f30e-aedf-42c6-870f-1d8828f65d73`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:04,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:04,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114329 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:06,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:06,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116679 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:08,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:08,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115887 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) (1964) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:09,856::__init__::225::IOProcess::(_processLogs) (1964) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:09,857::__init__::225::IOProcess::(_processLogs) DEBUG|(1964) Building response >Thread-21::DEBUG::2014-08-22 10:00:09,858::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:09,858::__init__::225::IOProcess::(_processLogs) (1964) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:09,864::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:00:09,865::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000228777 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:09,865::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:09,866::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) (1965) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:09,867::__init__::225::IOProcess::(_processLogs) (1965) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:09,868::__init__::225::IOProcess::(_processLogs) DEBUG|(1965) Building response >Thread-17::DEBUG::2014-08-22 10:00:09,869::__init__::225::IOProcess::(_processLogs) (1965) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:09,869::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:09,869::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:00:10,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:10,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118533 s, 86.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:11,171::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:11,171::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) (1966) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:11,172::__init__::225::IOProcess::(_processLogs) (1966) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:11,173::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1966) Building response >Thread-17::DEBUG::2014-08-22 10:00:11,173::__init__::225::IOProcess::(_processLogs) (1966) Queuing response >Thread-22::DEBUG::2014-08-22 10:00:11,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:11,174::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:11,180::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:00:11,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021663 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:11,181::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) (1967) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:11,182::__init__::225::IOProcess::(_processLogs) (1967) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:11,183::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1967) Building response >Thread-17::DEBUG::2014-08-22 10:00:11,183::__init__::225::IOProcess::(_processLogs) (1967) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:11,184::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:11,184::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:11,184::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:11,184::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:00:11,184::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:11,184::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) (1968) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) (1968) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) (1968) Building response >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) (1968) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:11,185::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:11,186::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:11,186::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:00:11,186::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:11,186::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:11,186::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:11,186::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) (1969) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) (1969) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) (1969) Building response >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) (1969) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:11,187::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:11,188::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:11,188::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:00:11,188::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:11,189::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:11,189::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:11,189::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:11,189::__init__::225::IOProcess::(_processLogs) (1970) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:11,189::__init__::225::IOProcess::(_processLogs) (1970) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:11,189::__init__::225::IOProcess::(_processLogs) (1970) Building response >Thread-17::DEBUG::2014-08-22 10:00:11,190::__init__::225::IOProcess::(_processLogs) (1970) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:11,190::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:11,190::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 10:00:12,197::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:00:12,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bbde5bd-66eb-4efa-b43e-dd8d6eec0e22`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:12,199::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:00:12,199::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:12,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bbde5bd-66eb-4efa-b43e-dd8d6eec0e22`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:12,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bbde5bd-66eb-4efa-b43e-dd8d6eec0e22`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:12,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:12,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:12,199::task::993::Storage.TaskManager.Task::(_decref) Task=`9bbde5bd-66eb-4efa-b43e-dd8d6eec0e22`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:00:12,296::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:00:12,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`e84366a3-e6e6-4b1b-84b7-c68a0162275e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:00:12,297::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:00:12,297::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cf8d9bfe-8685-41df-ae79-381354e6ea54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:00:12,297::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:00:12,297::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:00:12,297::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`cf8d9bfe-8685-41df-ae79-381354e6ea54`::Granted request >Thread-46::DEBUG::2014-08-22 10:00:12,298::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e84366a3-e6e6-4b1b-84b7-c68a0162275e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:00:12,298::task::993::Storage.TaskManager.Task::(_decref) Task=`e84366a3-e6e6-4b1b-84b7-c68a0162275e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:00:12,300::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:12,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`e84366a3-e6e6-4b1b-84b7-c68a0162275e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:12,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`e84366a3-e6e6-4b1b-84b7-c68a0162275e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:00:12,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:00:12,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:00:12,300::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:00:12,300::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:00:12,300::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:00:12,300::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:00:12,300::task::993::Storage.TaskManager.Task::(_decref) Task=`e84366a3-e6e6-4b1b-84b7-c68a0162275e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:12,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:12,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116353 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:14,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:14,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113929 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:16,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:16,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120282 s, 85.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:00:18,354::task::595::Storage.TaskManager.Task::(_updateState) Task=`ddc9fda9-56b2-4f36-9e8a-f15e883f324d`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:18,355::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:00:18,355::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000228777', 'lastCheck': '8.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021663', 'lastCheck': '7.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:18,355::task::1191::Storage.TaskManager.Task::(prepare) Task=`ddc9fda9-56b2-4f36-9e8a-f15e883f324d`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000228777', 'lastCheck': '8.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021663', 'lastCheck': '7.2', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:18,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`ddc9fda9-56b2-4f36-9e8a-f15e883f324d`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:18,355::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:18,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:18,355::task::993::Storage.TaskManager.Task::(_decref) Task=`ddc9fda9-56b2-4f36-9e8a-f15e883f324d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:18,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:18,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116244 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:19,869::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:19,869::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:19,869::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:19,869::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:19,869::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:19,870::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:19,870::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:19,870::__init__::225::IOProcess::(_processLogs) (1971) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:19,870::__init__::225::IOProcess::(_processLogs) (1971) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:19,870::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1971) Building response >Thread-21::DEBUG::2014-08-22 10:00:19,871::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:19,871::__init__::225::IOProcess::(_processLogs) (1971) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:19,878::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:19,879::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:00:19,879::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000153147 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:19,880::__init__::225::IOProcess::(_processLogs) (1972) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:19,881::__init__::225::IOProcess::(_processLogs) (1972) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:19,881::__init__::225::IOProcess::(_processLogs) (1972) Building response >Thread-17::DEBUG::2014-08-22 10:00:19,881::__init__::225::IOProcess::(_processLogs) (1972) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:19,881::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:19,881::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:00:20,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:20,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011681 s, 87.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:21,190::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:21,190::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) (1973) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:21,191::__init__::225::IOProcess::(_processLogs) (1973) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:21,192::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1973) Building response >Thread-17::DEBUG::2014-08-22 10:00:21,192::__init__::225::IOProcess::(_processLogs) (1973) Queuing response >Thread-22::DEBUG::2014-08-22 10:00:21,192::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:21,193::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:21,199::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:00:21,200::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000203533 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:21,200::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:21,200::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) (1974) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:21,201::__init__::225::IOProcess::(_processLogs) (1974) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:21,202::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(1974) Building response >Thread-17::DEBUG::2014-08-22 10:00:21,202::__init__::225::IOProcess::(_processLogs) (1974) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:21,202::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:21,203::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:21,203::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:21,203::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:00:21,203::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:21,203::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) (1975) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) (1975) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) (1975) Building response >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) (1975) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:21,204::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:21,205::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:21,205::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:00:21,205::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:21,205::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:21,205::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:21,205::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) (1976) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) (1976) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) (1976) Building response >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) (1976) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:00:21,206::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:21,207::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:21,207::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) (1977) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) (1977) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) (1977) Building response >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) (1977) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:21,208::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 10:00:22,351::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:00:22,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b3a88e9-a743-4c7e-9a7e-fcaaa8155ae6`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:22,351::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:00:22,352::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:22,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b3a88e9-a743-4c7e-9a7e-fcaaa8155ae6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:22,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b3a88e9-a743-4c7e-9a7e-fcaaa8155ae6`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:22,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:22,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:22,352::task::993::Storage.TaskManager.Task::(_decref) Task=`9b3a88e9-a743-4c7e-9a7e-fcaaa8155ae6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:00:22,403::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:00:22,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb4cc928-aba7-4f52-bf3b-76eb33f65452`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:00:22,403::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:00:22,404::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0b11a3f6-81bf-49c3-8079-c9886d90564c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:00:22,404::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:00:22,404::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:00:22,404::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0b11a3f6-81bf-49c3-8079-c9886d90564c`::Granted request >Thread-46::DEBUG::2014-08-22 10:00:22,404::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fb4cc928-aba7-4f52-bf3b-76eb33f65452`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:00:22,404::task::993::Storage.TaskManager.Task::(_decref) Task=`fb4cc928-aba7-4f52-bf3b-76eb33f65452`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:00:22,406::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:22,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb4cc928-aba7-4f52-bf3b-76eb33f65452`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:22,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb4cc928-aba7-4f52-bf3b-76eb33f65452`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:00:22,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:00:22,407::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:00:22,407::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:00:22,407::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:00:22,407::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:00:22,407::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:00:22,407::task::993::Storage.TaskManager.Task::(_decref) Task=`fb4cc928-aba7-4f52-bf3b-76eb33f65452`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:22,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:23,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116203 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:25,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:25,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115773 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:27,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:27,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114438 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:29,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:29,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115869 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:29,883::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:29,883::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:29,883::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:29,884::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:00:29,885::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:29,885::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:29,891::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:00:29,892::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00021607 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:29,892::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) (1978) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) (1978) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) (1978) Building response >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) (1978) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:29,894::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) (1979) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) (1979) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:29,895::__init__::225::IOProcess::(_processLogs) (1979) Building response >Thread-17::DEBUG::2014-08-22 10:00:29,896::__init__::225::IOProcess::(_processLogs) (1979) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:29,896::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:29,897::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:00:31,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:31,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117532 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:31,217::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:31,218::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:31,219::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:31,219::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:31,220::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:31,220::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:31,221::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:31,222::__init__::225::IOProcess::(_processLogs) (1980) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:31,222::__init__::225::IOProcess::(_processLogs) (1980) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:31,225::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|(1980) Building response >Thread-22::DEBUG::2014-08-22 10:00:31,226::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:31,226::__init__::225::IOProcess::(_processLogs) (1980) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:31,233::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:31,234::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:00:31,235::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000162838 s, 3.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:31,236::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:31,237::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:31,239::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:31,240::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:31,240::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:31,241::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:31,241::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:31,242::__init__::225::IOProcess::(_processLogs) (1981) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:31,242::__init__::225::IOProcess::(_processLogs) (1981) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:31,242::__init__::225::IOProcess::(_processLogs) (1981) Building response >Thread-17::DEBUG::2014-08-22 10:00:31,243::__init__::225::IOProcess::(_processLogs) (1981) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:31,243::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:31,244::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:31,244::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:31,245::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:00:31,245::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:31,246::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:31,247::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:31,247::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:31,248::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:31,248::__init__::225::IOProcess::(_processLogs) (1982) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:31,248::__init__::225::IOProcess::(_processLogs) (1982) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:31,249::__init__::225::IOProcess::(_processLogs) (1982) Building response >Thread-17::DEBUG::2014-08-22 10:00:31,249::__init__::225::IOProcess::(_processLogs) (1982) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:31,250::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:31,250::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:31,251::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:31,251::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:00:31,252::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:31,252::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:31,253::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:31,253::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:31,254::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:31,254::__init__::225::IOProcess::(_processLogs) (1983) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:31,255::__init__::225::IOProcess::(_processLogs) (1983) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:31,255::__init__::225::IOProcess::(_processLogs) (1983) Building response >Thread-17::DEBUG::2014-08-22 10:00:31,255::__init__::225::IOProcess::(_processLogs) (1983) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:31,256::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:31,256::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:31,257::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:31,257::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:00:31,258::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:31,258::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:31,258::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:31,259::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:31,259::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:31,260::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|(1984) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:31,261::__init__::225::IOProcess::(_processLogs) (1984) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:31,262::__init__::225::IOProcess::(_processLogs) (1984) Building response >Thread-17::DEBUG::2014-08-22 10:00:31,262::__init__::225::IOProcess::(_processLogs) (1984) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:31,262::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:31,263::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-47::DEBUG::2014-08-22 10:00:32,466::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:00:32,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`347da756-bdbb-46a4-a01e-6d3f5963cb14`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:32,468::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:00:32,469::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:32,470::task::1191::Storage.TaskManager.Task::(prepare) Task=`347da756-bdbb-46a4-a01e-6d3f5963cb14`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:32,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`347da756-bdbb-46a4-a01e-6d3f5963cb14`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:32,470::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:32,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:32,471::task::993::Storage.TaskManager.Task::(_decref) Task=`347da756-bdbb-46a4-a01e-6d3f5963cb14`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:00:32,655::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:00:32,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`a764bb8c-e454-47a7-bf81-9067e691be29`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:00:32,657::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:00:32,658::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bd7477d8-d8ed-444b-b52d-d194c2c879f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:00:32,658::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:00:32,659::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:00:32,659::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bd7477d8-d8ed-444b-b52d-d194c2c879f5`::Granted request >Thread-46::DEBUG::2014-08-22 10:00:32,661::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a764bb8c-e454-47a7-bf81-9067e691be29`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:00:32,661::task::993::Storage.TaskManager.Task::(_decref) Task=`a764bb8c-e454-47a7-bf81-9067e691be29`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:00:32,671::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:32,671::task::1191::Storage.TaskManager.Task::(prepare) Task=`a764bb8c-e454-47a7-bf81-9067e691be29`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:32,672::task::595::Storage.TaskManager.Task::(_updateState) Task=`a764bb8c-e454-47a7-bf81-9067e691be29`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:00:32,672::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:00:32,673::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:00:32,673::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:00:32,674::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:00:32,674::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:00:32,675::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:00:32,676::task::993::Storage.TaskManager.Task::(_decref) Task=`a764bb8c-e454-47a7-bf81-9067e691be29`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:33,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:33,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118556 s, 86.4 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:00:33,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7dc8890-f550-4a2e-8853-248ce916c859`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:33,531::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:00:33,531::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00021607', 'lastCheck': '3.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000162838', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:33,531::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7dc8890-f550-4a2e-8853-248ce916c859`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00021607', 'lastCheck': '3.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000162838', 'lastCheck': '2.3', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:33,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7dc8890-f550-4a2e-8853-248ce916c859`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:33,531::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:33,531::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:33,531::task::993::Storage.TaskManager.Task::(_decref) Task=`c7dc8890-f550-4a2e-8853-248ce916c859`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:35,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:35,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119632 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:37,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:37,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114994 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:39,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:39,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112778 s, 90.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:39,897::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:39,897::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:39,897::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:39,898::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:00:39,898::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:39,899::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:39,905::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:00:39,906::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000166213 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:39,906::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:39,907::__init__::225::IOProcess::(_processLogs) (1985) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:39,907::__init__::225::IOProcess::(_processLogs) (1985) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:39,907::__init__::225::IOProcess::(_processLogs) (1985) Building response >Thread-17::DEBUG::2014-08-22 10:00:39,907::__init__::225::IOProcess::(_processLogs) (1985) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) (1979) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:39,908::__init__::225::IOProcess::(_processLogs) (1986) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:39,909::__init__::225::IOProcess::(_processLogs) (1986) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:39,909::__init__::225::IOProcess::(_processLogs) (1986) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:39,910::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:39,910::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:41,261::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) (1987) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) (1987) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) (1987) Building response >Thread-17::DEBUG::2014-08-22 10:00:41,262::__init__::225::IOProcess::(_processLogs) (1987) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:41,263::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:00:41,263::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:00:41,269::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000251951 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:41,270::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:41,270::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:41,270::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:41,270::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:41,271::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:41,271::__init__::225::IOProcess::(_processLogs) (1988) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:41,271::__init__::225::IOProcess::(_processLogs) (1988) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:41,271::__init__::225::IOProcess::(_processLogs) (1988) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:41,272::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:41,272::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:41,272::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:00:41,272::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) (1989) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) (1989) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) (1989) Building response >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) (1989) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:41,273::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:41,274::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:41,274::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:00:41,274::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:41,274::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:41,274::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) (1990) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) (1990) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) (1990) Building response >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) (1990) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:00:41,275::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:41,276::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:41,276::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:41,276::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:41,276::__init__::225::IOProcess::(_processLogs) (1991) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:41,276::__init__::225::IOProcess::(_processLogs) (1991) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:41,277::__init__::225::IOProcess::(_processLogs) (1991) Building response >Thread-17::DEBUG::2014-08-22 10:00:41,277::__init__::225::IOProcess::(_processLogs) (1991) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:41,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:41,277::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:00:41,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:41,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114876 s, 89.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:00:42,728::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:00:42,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`e609deb7-2aa4-4149-bf87-b383c2dcc72b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:42,729::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:00:42,729::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:42,729::task::1191::Storage.TaskManager.Task::(prepare) Task=`e609deb7-2aa4-4149-bf87-b383c2dcc72b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:42,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`e609deb7-2aa4-4149-bf87-b383c2dcc72b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:42,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:42,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:42,730::task::993::Storage.TaskManager.Task::(_decref) Task=`e609deb7-2aa4-4149-bf87-b383c2dcc72b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:00:42,763::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:00:42,763::task::595::Storage.TaskManager.Task::(_updateState) Task=`0715e54a-22cd-4792-96fe-62a779f502d8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:00:42,763::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:00:42,763::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5437026b-8fdb-460c-9a4c-5625d19e8383`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:00:42,763::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:00:42,764::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:00:42,764::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5437026b-8fdb-460c-9a4c-5625d19e8383`::Granted request >Thread-46::DEBUG::2014-08-22 10:00:42,764::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0715e54a-22cd-4792-96fe-62a779f502d8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:00:42,764::task::993::Storage.TaskManager.Task::(_decref) Task=`0715e54a-22cd-4792-96fe-62a779f502d8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:00:42,766::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:42,766::task::1191::Storage.TaskManager.Task::(prepare) Task=`0715e54a-22cd-4792-96fe-62a779f502d8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:42,766::task::595::Storage.TaskManager.Task::(_updateState) Task=`0715e54a-22cd-4792-96fe-62a779f502d8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:00:42,766::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:00:42,766::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:00:42,766::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:00:42,766::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:00:42,767::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:00:42,767::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:00:42,767::task::993::Storage.TaskManager.Task::(_decref) Task=`0715e54a-22cd-4792-96fe-62a779f502d8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:43,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:43,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118283 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:45,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:45,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114865 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:47,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:47,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117766 s, 87.0 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:00:48,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf1c273e-418a-4410-98b7-6eb384689eff`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:48,803::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:00:48,803::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000166213', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000251951', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:48,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf1c273e-418a-4410-98b7-6eb384689eff`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000166213', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000251951', 'lastCheck': '7.5', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:00:48,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf1c273e-418a-4410-98b7-6eb384689eff`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:48,803::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:48,803::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:48,803::task::993::Storage.TaskManager.Task::(_decref) Task=`cf1c273e-418a-4410-98b7-6eb384689eff`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:49,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:49,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116927 s, 87.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) (1992) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:49,911::__init__::225::IOProcess::(_processLogs) (1992) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:49,912::__init__::225::IOProcess::(_processLogs) (1992) Queuing response >Thread-21::DEBUG::2014-08-22 10:00:49,913::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:49,913::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:49,919::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:00:49,920::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178329 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) (1993) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:49,921::__init__::225::IOProcess::(_processLogs) (1993) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:49,922::__init__::225::IOProcess::(_processLogs) (1993) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:49,923::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:49,923::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) (1994) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:51,277::__init__::225::IOProcess::(_processLogs) (1994) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:51,278::__init__::225::IOProcess::(_processLogs) (1994) Queuing response >Thread-22::DEBUG::2014-08-22 10:00:51,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:51,278::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:51,285::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:00:51,286::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000215615 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) (1995) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:51,287::__init__::225::IOProcess::(_processLogs) (1995) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:51,288::__init__::225::IOProcess::(_processLogs) (1995) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:51,288::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:51,289::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:00:51,289::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:00:51,289::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:51,289::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) (1996) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) (1996) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) (1996) Building response >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) (1996) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:00:51,290::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:51,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:51,291::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:51,291::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:51,291::__init__::225::IOProcess::(_processLogs) (1997) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:51,291::__init__::225::IOProcess::(_processLogs) (1997) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) (1997) Building response >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) (1997) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:51,292::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:51,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:51,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:51,293::__init__::225::IOProcess::(_processLogs) (1998) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:00:51,293::__init__::225::IOProcess::(_processLogs) (1998) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:00:51,294::__init__::225::IOProcess::(_processLogs) (1998) Building response >Thread-17::DEBUG::2014-08-22 10:00:51,294::__init__::225::IOProcess::(_processLogs) (1998) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:51,294::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:51,294::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:00:51,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:51,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218305 s, 46.9 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:00:52,815::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:00:52,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`be9b72bb-77b1-47eb-93a2-21b11e291404`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:00:52,816::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:00:52,816::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:52,816::task::1191::Storage.TaskManager.Task::(prepare) Task=`be9b72bb-77b1-47eb-93a2-21b11e291404`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:00:52,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`be9b72bb-77b1-47eb-93a2-21b11e291404`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:00:52,816::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:00:52,816::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:00:52,816::task::993::Storage.TaskManager.Task::(_decref) Task=`be9b72bb-77b1-47eb-93a2-21b11e291404`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:00:52,859::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:00:52,859::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6d6881b-03d2-43d0-8a41-f442ee08c967`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:00:52,859::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:00:52,859::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`addc1f5b-1fc5-4420-bcef-1fb200f01997`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:00:52,859::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:00:52,860::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:00:52,860::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`addc1f5b-1fc5-4420-bcef-1fb200f01997`::Granted request >Thread-46::DEBUG::2014-08-22 10:00:52,861::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d6d6881b-03d2-43d0-8a41-f442ee08c967`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:00:52,861::task::993::Storage.TaskManager.Task::(_decref) Task=`d6d6881b-03d2-43d0-8a41-f442ee08c967`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:00:52,863::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:52,863::task::1191::Storage.TaskManager.Task::(prepare) Task=`d6d6881b-03d2-43d0-8a41-f442ee08c967`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:00:52,863::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6d6881b-03d2-43d0-8a41-f442ee08c967`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:00:52,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:00:52,863::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:00:52,863::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:00:52,863::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:00:52,863::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:00:52,863::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:00:52,863::task::993::Storage.TaskManager.Task::(_decref) Task=`d6d6881b-03d2-43d0-8a41-f442ee08c967`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:00:53,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:53,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113935 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:55,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:55,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119254 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:00:57,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:00:57,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120024 s, 85.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:59,923::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:59,923::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:00:59,923::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:59,924::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:59,924::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:59,924::__init__::225::IOProcess::(_processLogs) (1999) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:59,924::__init__::225::IOProcess::(_processLogs) (1999) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:59,924::__init__::225::IOProcess::(_processLogs) (1999) Queuing response >Thread-21::DEBUG::2014-08-22 10:00:59,925::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:00:59,925::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:59,932::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:00:59,933::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000183281 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:00:59,933::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:00:59,933::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:00:59,933::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:00:59,934::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:00:59,934::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:00:59,934::__init__::225::IOProcess::(_processLogs) (2000) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:00:59,934::__init__::225::IOProcess::(_processLogs) (2000) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:00:59,935::__init__::225::IOProcess::(_processLogs) (2000) Queuing response >Thread-17::DEBUG::2014-08-22 10:00:59,935::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:00:59,935::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:01:00,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:00,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115324 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) (2001) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) (2001) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:01,294::__init__::225::IOProcess::(_processLogs) (2001) Building response >Thread-17::DEBUG::2014-08-22 10:01:01,295::__init__::225::IOProcess::(_processLogs) (2001) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:01,295::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:01:01,296::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:01:01,302::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000280551 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:01,303::__init__::225::IOProcess::(_processLogs) (2002) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:01,304::__init__::225::IOProcess::(_processLogs) (2002) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:01,304::__init__::225::IOProcess::(_processLogs) (2002) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:01,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:01,305::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:01,305::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:01:01,305::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) (2003) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) (2003) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) (2003) Building response >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) (2003) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:01,306::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) (2004) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) (2004) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) (2004) Building response >Thread-17::DEBUG::2014-08-22 10:01:01,307::__init__::225::IOProcess::(_processLogs) (2004) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:01,308::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:01,308::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:01,308::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:01:01,308::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:01,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:01,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) (2005) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) (2005) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) (2005) Building response >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) (2005) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:01,309::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:01:02,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:02,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178812 s, 57.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:01:02,913::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:01:02,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`70993bce-e6e5-407a-8ab8-aeec5450b389`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:02,915::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:01:02,916::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:02,916::task::1191::Storage.TaskManager.Task::(prepare) Task=`70993bce-e6e5-407a-8ab8-aeec5450b389`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:02,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`70993bce-e6e5-407a-8ab8-aeec5450b389`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:02,917::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:02,918::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:02,918::task::993::Storage.TaskManager.Task::(_decref) Task=`70993bce-e6e5-407a-8ab8-aeec5450b389`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:01:02,955::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:01:02,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`7bfa6a54-f42a-4696-bdd4-5dc20ff25cc0`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:01:02,956::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:01:02,958::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bc282fef-80f7-46a3-875c-4be5ea092f9b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:01:02,958::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:01:02,959::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:01:02,959::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bc282fef-80f7-46a3-875c-4be5ea092f9b`::Granted request >Thread-46::DEBUG::2014-08-22 10:01:02,960::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7bfa6a54-f42a-4696-bdd4-5dc20ff25cc0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:01:02,961::task::993::Storage.TaskManager.Task::(_decref) Task=`7bfa6a54-f42a-4696-bdd4-5dc20ff25cc0`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:01:02,966::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:02,967::task::1191::Storage.TaskManager.Task::(prepare) Task=`7bfa6a54-f42a-4696-bdd4-5dc20ff25cc0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:02,968::task::595::Storage.TaskManager.Task::(_updateState) Task=`7bfa6a54-f42a-4696-bdd4-5dc20ff25cc0`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:01:02,968::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:01:02,968::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:01:02,968::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:01:02,968::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:01:02,969::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:01:02,969::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:01:02,969::task::993::Storage.TaskManager.Task::(_decref) Task=`7bfa6a54-f42a-4696-bdd4-5dc20ff25cc0`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 10:01:03,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`d43157c7-a4b0-442b-8675-45f1243ad49e`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:03,992::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:01:03,992::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183281', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000280551', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:01:03,992::task::1191::Storage.TaskManager.Task::(prepare) Task=`d43157c7-a4b0-442b-8675-45f1243ad49e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000183281', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000280551', 'lastCheck': '2.7', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:01:03,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`d43157c7-a4b0-442b-8675-45f1243ad49e`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:03,992::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:03,992::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:03,992::task::993::Storage.TaskManager.Task::(_decref) Task=`d43157c7-a4b0-442b-8675-45f1243ad49e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:01:04,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:04,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113033 s, 90.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:06,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:06,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113548 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:08,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:08,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117202 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) (2006) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:09,936::__init__::225::IOProcess::(_processLogs) (2006) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:09,937::__init__::225::IOProcess::(_processLogs) DEBUG|(2006) Building response >Thread-21::DEBUG::2014-08-22 10:01:09,938::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:09,938::__init__::225::IOProcess::(_processLogs) (2006) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:09,944::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:01:09,945::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000176092 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:09,945::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) (2007) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:09,947::__init__::225::IOProcess::(_processLogs) (2007) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:09,948::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2007) Building response >Thread-17::DEBUG::2014-08-22 10:01:09,949::__init__::225::IOProcess::(_processLogs) (2007) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:09,949::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:09,949::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:01:10,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:10,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139011 s, 73.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:11,310::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:11,310::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:11,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:11,310::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:11,311::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:11,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:11,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:11,311::__init__::225::IOProcess::(_processLogs) (2008) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:11,311::__init__::225::IOProcess::(_processLogs) (2008) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:11,311::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2008) Building response >Thread-22::DEBUG::2014-08-22 10:01:11,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:11,312::__init__::225::IOProcess::(_processLogs) (2008) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:11,319::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:01:11,320::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000185028 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:11,320::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:11,321::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:11,322::__init__::225::IOProcess::(_processLogs) (2009) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:11,322::__init__::225::IOProcess::(_processLogs) (2009) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:11,322::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2009) Building response >Thread-17::DEBUG::2014-08-22 10:01:11,323::__init__::225::IOProcess::(_processLogs) (2009) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:11,323::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:11,323::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:11,323::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:11,323::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:01:11,323::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:11,324::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:11,324::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:11,324::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:11,324::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:11,324::__init__::225::IOProcess::(_processLogs) (2010) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:11,324::__init__::225::IOProcess::(_processLogs) (2010) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) (2010) Building response >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) (2010) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:01:11,325::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) (2011) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) (2011) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) (2011) Building response >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) (2011) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:11,326::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:11,328::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:11,328::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:01:11,328::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:11,328::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:11,328::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) (2012) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) (2012) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) (2012) Building response >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) (2012) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:11,329::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:01:12,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:12,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116192 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:01:13,056::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:01:13,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`d52a824d-c1a1-48e5-8f9e-45ff081d5943`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:13,057::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:01:13,057::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:13,057::task::1191::Storage.TaskManager.Task::(prepare) Task=`d52a824d-c1a1-48e5-8f9e-45ff081d5943`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:13,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`d52a824d-c1a1-48e5-8f9e-45ff081d5943`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:13,057::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:13,057::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:13,057::task::993::Storage.TaskManager.Task::(_decref) Task=`d52a824d-c1a1-48e5-8f9e-45ff081d5943`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:01:13,100::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:01:13,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ba72764-31a6-407f-a88e-59df12264df4`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:01:13,100::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:01:13,100::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`95877832-44de-4264-9173-7a986d987548`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:01:13,100::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:01:13,101::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:01:13,101::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`95877832-44de-4264-9173-7a986d987548`::Granted request >Thread-46::DEBUG::2014-08-22 10:01:13,101::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0ba72764-31a6-407f-a88e-59df12264df4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:01:13,101::task::993::Storage.TaskManager.Task::(_decref) Task=`0ba72764-31a6-407f-a88e-59df12264df4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:01:13,103::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:13,103::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ba72764-31a6-407f-a88e-59df12264df4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:13,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ba72764-31a6-407f-a88e-59df12264df4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:01:13,103::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:01:13,103::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:01:13,103::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:01:13,103::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:01:13,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:01:13,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:01:13,104::task::993::Storage.TaskManager.Task::(_decref) Task=`0ba72764-31a6-407f-a88e-59df12264df4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:01:14,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:14,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116899 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:16,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:16,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114522 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:18,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:18,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116636 s, 87.8 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:01:19,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cf9690c-ba9f-44c2-96af-938119f0a53a`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:19,172::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:01:19,173::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000176092', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185028', 'lastCheck': '7.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:01:19,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`9cf9690c-ba9f-44c2-96af-938119f0a53a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000176092', 'lastCheck': '9.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185028', 'lastCheck': '7.8', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:01:19,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cf9690c-ba9f-44c2-96af-938119f0a53a`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:19,173::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:19,173::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:19,173::task::993::Storage.TaskManager.Task::(_decref) Task=`9cf9690c-ba9f-44c2-96af-938119f0a53a`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:01:19,949::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:19,949::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:19,950::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:19,950::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:19,951::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 10:01:19,951::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:19,952::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:19,958::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 10:01:19,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187169 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:19,959::__init__::225::IOProcess::(_processLogs) (2013) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:19,960::__init__::225::IOProcess::(_processLogs) (2013) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:19,960::__init__::225::IOProcess::(_processLogs) (2013) Building response >Thread-17::DEBUG::2014-08-22 10:01:19,960::__init__::225::IOProcess::(_processLogs) (2013) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:19,960::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:19,960::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) (2014) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:19,961::__init__::225::IOProcess::(_processLogs) (2014) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:19,962::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2014) Building response >Thread-17::DEBUG::2014-08-22 10:01:19,963::__init__::225::IOProcess::(_processLogs) (2014) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:19,963::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:19,963::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:01:20,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:20,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114131 s, 89.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:21,329::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:21,329::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:21,329::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:21,329::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:21,330::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:21,330::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:21,330::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:21,330::__init__::225::IOProcess::(_processLogs) (2015) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:21,330::__init__::225::IOProcess::(_processLogs) (2015) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:21,330::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2015) Building response >Thread-17::DEBUG::2014-08-22 10:01:21,331::__init__::225::IOProcess::(_processLogs) (2015) Queuing response >Thread-22::DEBUG::2014-08-22 10:01:21,331::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:21,332::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:21,338::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:01:21,338::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000199269 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:21,339::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:21,340::__init__::225::IOProcess::(_processLogs) (2016) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:21,340::__init__::225::IOProcess::(_processLogs) (2016) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:21,341::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2016) Building response >Thread-17::DEBUG::2014-08-22 10:01:21,341::__init__::225::IOProcess::(_processLogs) (2016) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:21,341::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:21,341::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:21,342::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:21,342::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:01:21,342::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:21,342::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) (2017) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) (2017) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) (2017) Building response >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) (2017) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:21,343::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:01:21,344::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:21,344::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:21,344::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:21,344::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:21,344::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:21,344::__init__::225::IOProcess::(_processLogs) (2018) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) (2018) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) (2018) Building response >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) (2018) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:01:21,345::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:21,346::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:21,346::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:21,346::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:21,346::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:21,346::__init__::225::IOProcess::(_processLogs) (2019) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:21,347::__init__::225::IOProcess::(_processLogs) (2019) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:21,347::__init__::225::IOProcess::(_processLogs) (2019) Building response >Thread-17::DEBUG::2014-08-22 10:01:21,347::__init__::225::IOProcess::(_processLogs) (2019) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:21,347::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:21,347::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:01:22,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:22,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116228 s, 88.1 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:01:23,153::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:01:23,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`6beb6c4a-f130-4ec1-b968-4a865b26ef6b`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:23,154::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:01:23,154::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:23,154::task::1191::Storage.TaskManager.Task::(prepare) Task=`6beb6c4a-f130-4ec1-b968-4a865b26ef6b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:23,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`6beb6c4a-f130-4ec1-b968-4a865b26ef6b`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:23,154::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:23,154::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:23,154::task::993::Storage.TaskManager.Task::(_decref) Task=`6beb6c4a-f130-4ec1-b968-4a865b26ef6b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:01:23,195::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:01:23,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a0803ef-8eff-4a1b-a246-39eabaa004ef`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:01:23,196::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:01:23,196::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a9f4d217-e51d-4ff6-9016-5a5b72ed1ded`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:01:23,196::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:01:23,196::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:01:23,196::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a9f4d217-e51d-4ff6-9016-5a5b72ed1ded`::Granted request >Thread-46::DEBUG::2014-08-22 10:01:23,197::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8a0803ef-8eff-4a1b-a246-39eabaa004ef`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:01:23,197::task::993::Storage.TaskManager.Task::(_decref) Task=`8a0803ef-8eff-4a1b-a246-39eabaa004ef`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:01:23,199::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:23,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a0803ef-8eff-4a1b-a246-39eabaa004ef`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:23,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a0803ef-8eff-4a1b-a246-39eabaa004ef`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:01:23,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:01:23,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:01:23,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:01:23,199::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:01:23,199::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:01:23,199::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:01:23,200::task::993::Storage.TaskManager.Task::(_decref) Task=`8a0803ef-8eff-4a1b-a246-39eabaa004ef`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:01:24,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:24,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116018 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:26,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:26,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113608 s, 90.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:28,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:28,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116687 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:29,963::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:29,963::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) (2020) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) (2020) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) (2020) Building response >Thread-17::DEBUG::2014-08-22 10:01:29,964::__init__::225::IOProcess::(_processLogs) (2020) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:29,965::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 10:01:29,965::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:29,966::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:01:29,972::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00034352 s, 1.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:29,973::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:29,973::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:29,973::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:29,973::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:29,973::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:29,973::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:29,974::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:29,974::__init__::225::IOProcess::(_processLogs) (2021) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:29,974::__init__::225::IOProcess::(_processLogs) (2021) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:29,974::__init__::225::IOProcess::(_processLogs) (2021) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:29,975::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:29,975::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:01:30,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:30,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120874 s, 84.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) (2022) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:31,347::__init__::225::IOProcess::(_processLogs) (2022) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:31,355::__init__::225::IOProcess::(_processLogs) (2022) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:31,356::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:01:31,356::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:01:31,371::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00466649 s, 122 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:31,371::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:31,371::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:31,372::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:31,372::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:31,372::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:31,372::__init__::225::IOProcess::(_processLogs) (2023) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:31,372::__init__::225::IOProcess::(_processLogs) (2023) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:31,374::__init__::225::IOProcess::(_processLogs) (2023) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:31,374::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:31,375::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:01:31,375::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:31,376::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:31,376::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:31,376::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) (2024) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) (2024) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) (2024) Building response >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) (2024) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:01:31,377::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:31,378::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:31,378::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:31,378::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:31,378::__init__::225::IOProcess::(_processLogs) (2025) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:31,378::__init__::225::IOProcess::(_processLogs) (2025) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:31,378::__init__::225::IOProcess::(_processLogs) (2025) Building response >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) (2025) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:31,379::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:31,380::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:31,380::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:31,380::__init__::225::IOProcess::(_processLogs) (2026) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:31,380::__init__::225::IOProcess::(_processLogs) (2026) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:31,380::__init__::225::IOProcess::(_processLogs) (2026) Building response >Thread-17::DEBUG::2014-08-22 10:01:31,381::__init__::225::IOProcess::(_processLogs) (2026) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:31,381::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:31,381::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:01:32,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:32,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260344 s, 39.3 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:01:33,248::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:01:33,248::task::595::Storage.TaskManager.Task::(_updateState) Task=`470277d1-b5ea-44fa-b726-0708cf78fef7`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:33,249::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:01:33,249::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:33,249::task::1191::Storage.TaskManager.Task::(prepare) Task=`470277d1-b5ea-44fa-b726-0708cf78fef7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:33,249::task::595::Storage.TaskManager.Task::(_updateState) Task=`470277d1-b5ea-44fa-b726-0708cf78fef7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:33,249::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:33,249::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:33,249::task::993::Storage.TaskManager.Task::(_decref) Task=`470277d1-b5ea-44fa-b726-0708cf78fef7`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:01:33,324::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:01:33,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`716030c1-0c9a-4c37-ab6d-6aeb4b3ecebd`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:01:33,325::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:01:33,325::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bdaf00a2-2514-4643-b29e-22c8ccbd4095`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:01:33,325::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:01:33,325::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:01:33,325::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`bdaf00a2-2514-4643-b29e-22c8ccbd4095`::Granted request >Thread-46::DEBUG::2014-08-22 10:01:33,325::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`716030c1-0c9a-4c37-ab6d-6aeb4b3ecebd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:01:33,326::task::993::Storage.TaskManager.Task::(_decref) Task=`716030c1-0c9a-4c37-ab6d-6aeb4b3ecebd`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:01:33,327::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:33,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`716030c1-0c9a-4c37-ab6d-6aeb4b3ecebd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107359502336', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:33,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`716030c1-0c9a-4c37-ab6d-6aeb4b3ecebd`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:01:33,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:01:33,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:01:33,328::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:01:33,328::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:01:33,328::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:01:33,328::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:01:33,328::task::993::Storage.TaskManager.Task::(_decref) Task=`716030c1-0c9a-4c37-ab6d-6aeb4b3ecebd`::ref 0 aborting False >Thread-47::DEBUG::2014-08-22 10:01:34,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d609cb9-7044-4f07-8206-3f7c0d26f1e3`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:34,352::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-08-22 10:01:34,352::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00034352', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00466649', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:01:34,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d609cb9-7044-4f07-8206-3f7c0d26f1e3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00034352', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00466649', 'lastCheck': '3.0', 'valid': True}} >Thread-47::DEBUG::2014-08-22 10:01:34,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d609cb9-7044-4f07-8206-3f7c0d26f1e3`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:34,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:34,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:34,352::task::993::Storage.TaskManager.Task::(_decref) Task=`9d609cb9-7044-4f07-8206-3f7c0d26f1e3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:01:34,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:35,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114075 s, 89.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:37,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:37,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112926 s, 90.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:39,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:39,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112141 s, 91.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) (2027) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:39,976::__init__::225::IOProcess::(_processLogs) (2027) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:39,977::__init__::225::IOProcess::(_processLogs) (2027) Queuing response >Thread-21::DEBUG::2014-08-22 10:01:39,978::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:39,978::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:39,984::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:01:39,985::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000208238 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) (2028) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:39,986::__init__::225::IOProcess::(_processLogs) (2028) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:39,987::__init__::225::IOProcess::(_processLogs) (2028) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:39,988::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:39,988::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:01:41,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:41,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115762 s, 88.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) (2029) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:41,381::__init__::225::IOProcess::(_processLogs) (2029) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:41,382::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2029) Building response >Thread-17::DEBUG::2014-08-22 10:01:41,383::__init__::225::IOProcess::(_processLogs) (2029) Queuing response >Thread-22::DEBUG::2014-08-22 10:01:41,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:41,383::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:41,389::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:01:41,390::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000215256 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) (2030) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:41,391::__init__::225::IOProcess::(_processLogs) (2030) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:41,392::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2030) Building response >Thread-17::DEBUG::2014-08-22 10:01:41,393::__init__::225::IOProcess::(_processLogs) (2030) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:41,393::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:41,393::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:41,393::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:41,393::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:01:41,393::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) (2031) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) (2031) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) (2031) Building response >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) (2031) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:41,394::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:41,395::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:41,395::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:01:41,395::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:41,395::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:41,395::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) (2032) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) (2032) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) (2032) Building response >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) (2032) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:41,396::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:01:41,397::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:41,397::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:41,397::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) (2033) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) (2033) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) (2033) Building response >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) (2033) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:41,398::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:01:43,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:43,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118276 s, 86.6 MB/s\n'; <rc> = 0 >Thread-47::DEBUG::2014-08-22 10:01:43,513::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-47::DEBUG::2014-08-22 10:01:43,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad408e2e-8e62-433b-a1b6-aaf794a09faa`::moving from state init -> state preparing >Thread-47::INFO::2014-08-22 10:01:43,514::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-47::INFO::2014-08-22 10:01:43,514::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:43,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad408e2e-8e62-433b-a1b6-aaf794a09faa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-47::DEBUG::2014-08-22 10:01:43,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad408e2e-8e62-433b-a1b6-aaf794a09faa`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-08-22 10:01:43,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-08-22 10:01:43,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-08-22 10:01:43,515::task::993::Storage.TaskManager.Task::(_decref) Task=`ad408e2e-8e62-433b-a1b6-aaf794a09faa`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:01:43,550::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:01:43,550::task::595::Storage.TaskManager.Task::(_updateState) Task=`c69dd7b2-9c93-4b72-b4a3-9636bbc61931`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:01:43,551::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:01:43,551::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f3129b7b-8de4-4124-a8d0-12d9343eb3bf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:01:43,551::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:01:43,551::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:01:43,551::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f3129b7b-8de4-4124-a8d0-12d9343eb3bf`::Granted request >Thread-46::DEBUG::2014-08-22 10:01:43,551::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c69dd7b2-9c93-4b72-b4a3-9636bbc61931`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:01:43,551::task::993::Storage.TaskManager.Task::(_decref) Task=`c69dd7b2-9c93-4b72-b4a3-9636bbc61931`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:01:43,553::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:43,553::task::1191::Storage.TaskManager.Task::(prepare) Task=`c69dd7b2-9c93-4b72-b4a3-9636bbc61931`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:43,554::task::595::Storage.TaskManager.Task::(_updateState) Task=`c69dd7b2-9c93-4b72-b4a3-9636bbc61931`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:01:43,554::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:01:43,554::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:01:43,554::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:01:43,554::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:01:43,554::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:01:43,554::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:01:43,554::task::993::Storage.TaskManager.Task::(_decref) Task=`c69dd7b2-9c93-4b72-b4a3-9636bbc61931`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:01:45,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:45,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121975 s, 84.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:47,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:47,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116322 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:49,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:49,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113727 s, 90.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:01:49,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`cccb0df3-f5b9-4110-b317-834aa2cd46c3`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:01:49,541::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:01:49,542::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208238', 'lastCheck': '9.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000215256', 'lastCheck': '8.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:01:49,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`cccb0df3-f5b9-4110-b317-834aa2cd46c3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000208238', 'lastCheck': '9.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000215256', 'lastCheck': '8.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:01:49,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`cccb0df3-f5b9-4110-b317-834aa2cd46c3`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:01:49,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:01:49,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:01:49,542::task::993::Storage.TaskManager.Task::(_decref) Task=`cccb0df3-f5b9-4110-b317-834aa2cd46c3`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:01:49,988::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:49,988::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:49,988::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:49,988::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:49,989::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:49,989::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:49,989::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:49,989::__init__::225::IOProcess::(_processLogs) (2034) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:49,989::__init__::225::IOProcess::(_processLogs) (2034) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:49,989::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2034) Building response >Thread-21::DEBUG::2014-08-22 10:01:49,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:49,990::__init__::225::IOProcess::(_processLogs) (2034) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:49,997::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:49,998::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:01:49,998::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000217468 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:49,999::__init__::225::IOProcess::(_processLogs) (2035) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:50,000::__init__::225::IOProcess::(_processLogs) (2035) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:50,000::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2035) Building response >Thread-17::DEBUG::2014-08-22 10:01:50,001::__init__::225::IOProcess::(_processLogs) (2035) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:50,001::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:50,001::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:01:51,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:51,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118815 s, 86.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:51,419::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:51,419::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:51,420::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:51,421::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:51,421::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:51,422::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:51,425::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-22::DEBUG::2014-08-22 10:01:51,426::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:01:51,426::__init__::225::IOProcess::(_processLogs) (2036) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:51,433::__init__::225::IOProcess::(_processLogs) (2036) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 10:01:51,434::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000220417 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:01:51,434::__init__::225::IOProcess::(_processLogs) (2036) Building response >Thread-17::DEBUG::2014-08-22 10:01:51,436::__init__::225::IOProcess::(_processLogs) (2036) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:51,436::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:51,437::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:51,438::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:51,439::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:01:51,439::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:51,440::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:51,441::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:51,441::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:51,442::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:51,443::__init__::225::IOProcess::(_processLogs) (2037) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:01:51,449::__init__::225::IOProcess::(_processLogs) (2037) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:01:51,450::__init__::225::IOProcess::(_processLogs) (2037) Building response >Thread-17::DEBUG::2014-08-22 10:01:51,451::__init__::225::IOProcess::(_processLogs) (2037) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:51,451::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:51,452::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:01:51,453::__init__::225::IOProcess::(_processLogs) (2037) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:51,453::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:01:51,454::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:51,454::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:51,454::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:51,455::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:51,455::__init__::225::IOProcess::(_processLogs) (2037) Got request for method 'statvfs'DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:51,456::__init__::225::IOProcess::(_processLogs) (2038) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:51,456::__init__::225::IOProcess::(_processLogs) (2038) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:51,457::__init__::225::IOProcess::(_processLogs) (2038) Building response >Thread-17::DEBUG::2014-08-22 10:01:51,457::__init__::225::IOProcess::(_processLogs) (2038) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:51,458::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:51,458::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:51,459::__init__::225::IOProcess::(_processLogs) (2037) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:51,459::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:01:51,460::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:51,460::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:51,461::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:51,462::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:51,463::__init__::225::IOProcess::(_processLogs) (2037) Got request for method 'statvfs'DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:51,464::__init__::225::IOProcess::(_processLogs) (2039) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:51,465::__init__::225::IOProcess::(_processLogs) (2039) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:51,465::__init__::225::IOProcess::(_processLogs) (2039) Building response >Thread-17::DEBUG::2014-08-22 10:01:51,466::__init__::225::IOProcess::(_processLogs) (2039) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:51,466::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:51,467::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:01:51,468::__init__::225::IOProcess::(_processLogs) (2037) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:01:51,469::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:01:51,469::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:01:51,470::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:01:51,471::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:01:51,471::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:01:51,472::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:01:51,472::__init__::225::IOProcess::(_processLogs) (2040) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:01:51,473::__init__::225::IOProcess::(_processLogs) (2040) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:01:51,474::__init__::225::IOProcess::(_processLogs) (2040) Building response >Thread-17::DEBUG::2014-08-22 10:01:51,475::__init__::225::IOProcess::(_processLogs) (2040) Queuing response >Thread-17::DEBUG::2014-08-22 10:01:51,476::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:01:51,479::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:01:53,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:53,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115783 s, 88.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:01:53,658::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:01:53,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9aacdac-17cf-4b29-8d9b-4dc53fd40993`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:01:53,659::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:01:53,659::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:01:53,659::task::1191::Storage.TaskManager.Task::(prepare) Task=`e9aacdac-17cf-4b29-8d9b-4dc53fd40993`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:01:53,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9aacdac-17cf-4b29-8d9b-4dc53fd40993`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:01:53,659::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:01:53,659::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:01:53,659::task::993::Storage.TaskManager.Task::(_decref) Task=`e9aacdac-17cf-4b29-8d9b-4dc53fd40993`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:01:53,691::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:01:53,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`179879a2-f133-4dbe-be52-8ab5c609a062`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:01:53,692::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:01:53,692::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`10c30d53-54a0-40f6-acbf-74a6d324ed96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:01:53,692::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:01:53,692::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:01:53,692::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`10c30d53-54a0-40f6-acbf-74a6d324ed96`::Granted request >Thread-46::DEBUG::2014-08-22 10:01:53,692::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`179879a2-f133-4dbe-be52-8ab5c609a062`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:01:53,692::task::993::Storage.TaskManager.Task::(_decref) Task=`179879a2-f133-4dbe-be52-8ab5c609a062`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:01:53,694::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:53,695::task::1191::Storage.TaskManager.Task::(prepare) Task=`179879a2-f133-4dbe-be52-8ab5c609a062`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:01:53,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`179879a2-f133-4dbe-be52-8ab5c609a062`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:01:53,695::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:01:53,695::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:01:53,695::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:01:53,695::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:01:53,695::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:01:53,695::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:01:53,695::task::993::Storage.TaskManager.Task::(_decref) Task=`179879a2-f133-4dbe-be52-8ab5c609a062`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:01:55,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:55,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011164 s, 91.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:57,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:57,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115607 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:01:59,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:01:59,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113239 s, 90.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) (2041) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:00,002::__init__::225::IOProcess::(_processLogs) (2041) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:00,003::__init__::225::IOProcess::(_processLogs) DEBUG|(2041) Building response >Thread-21::DEBUG::2014-08-22 10:02:00,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:00,004::__init__::225::IOProcess::(_processLogs) (2041) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:00,010::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:02:00,011::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00024877 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:00,011::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:00,013::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:00,013::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:00,013::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:00,013::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:00,014::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:00,014::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:00,014::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:00,015::__init__::225::IOProcess::(_processLogs) (2042) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:00,015::__init__::225::IOProcess::(_processLogs) (2042) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:00,015::__init__::225::IOProcess::(_processLogs) (2042) Building response >Thread-17::DEBUG::2014-08-22 10:02:00,015::__init__::225::IOProcess::(_processLogs) (2042) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:00,015::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:00,015::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) (2043) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:01,476::__init__::225::IOProcess::(_processLogs) (2043) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:01,478::__init__::225::IOProcess::(_processLogs) DEBUG|(2043) Building response >Thread-17::DEBUG::2014-08-22 10:02:01,478::__init__::225::IOProcess::(_processLogs) (2043) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:01,479::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:02:01,479::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:02:01,479::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:02:01,490::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000286635 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:01,491::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:01,491::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:01,491::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:01,491::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) (2044) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) (2044) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) (2044) Building response >Thread-17::DEBUG::2014-08-22 10:02:01,492::__init__::225::IOProcess::(_processLogs) (2044) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:01,493::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:01,493::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:02:01,494::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:01,494::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:01,495::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:01,495::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) (2045) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) (2045) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) (2045) Building response >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) (2045) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:01,496::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:01,497::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:01,497::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:01,497::__init__::225::IOProcess::(_processLogs) (2046) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:01,497::__init__::225::IOProcess::(_processLogs) (2046) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:01,498::__init__::225::IOProcess::(_processLogs) (2046) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:01,499::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:01,499::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:01,499::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:01,499::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) (2047) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:01,500::__init__::225::IOProcess::(_processLogs) (2047) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:01,505::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|(2047) Building response >Thread-17::DEBUG::2014-08-22 10:02:01,505::__init__::225::IOProcess::(_processLogs) (2047) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:01,505::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:02:01,506::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:02:01,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:01,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011613 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:03,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:03,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115243 s, 88.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:03,794::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:02:03,795::task::595::Storage.TaskManager.Task::(_updateState) Task=`29054c96-6e39-414b-9190-62f16422f88d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:03,795::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:02:03,795::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:03,795::task::1191::Storage.TaskManager.Task::(prepare) Task=`29054c96-6e39-414b-9190-62f16422f88d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:03,795::task::595::Storage.TaskManager.Task::(_updateState) Task=`29054c96-6e39-414b-9190-62f16422f88d`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:03,795::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:03,796::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:03,796::task::993::Storage.TaskManager.Task::(_decref) Task=`29054c96-6e39-414b-9190-62f16422f88d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:02:03,827::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:02:03,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fd33f78-7468-4835-bb73-9d14cabb9a10`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:02:03,827::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:02:03,828::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4463c808-521a-4948-8104-5ed73d03c8c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:02:03,828::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:02:03,828::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:02:03,828::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4463c808-521a-4948-8104-5ed73d03c8c1`::Granted request >Thread-46::DEBUG::2014-08-22 10:02:03,828::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0fd33f78-7468-4835-bb73-9d14cabb9a10`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:02:03,828::task::993::Storage.TaskManager.Task::(_decref) Task=`0fd33f78-7468-4835-bb73-9d14cabb9a10`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:02:03,830::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:03,830::task::1191::Storage.TaskManager.Task::(prepare) Task=`0fd33f78-7468-4835-bb73-9d14cabb9a10`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:03,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fd33f78-7468-4835-bb73-9d14cabb9a10`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:02:03,830::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:02:03,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:02:03,831::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:02:03,831::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:02:03,831::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:02:03,831::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:02:03,831::task::993::Storage.TaskManager.Task::(_decref) Task=`0fd33f78-7468-4835-bb73-9d14cabb9a10`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 10:02:04,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`1757b0f9-3851-41cc-af19-fd05cecdcb1a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:04,817::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:02:04,817::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00024877', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000286635', 'lastCheck': '3.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:04,817::task::1191::Storage.TaskManager.Task::(prepare) Task=`1757b0f9-3851-41cc-af19-fd05cecdcb1a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00024877', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000286635', 'lastCheck': '3.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:04,818::task::595::Storage.TaskManager.Task::(_updateState) Task=`1757b0f9-3851-41cc-af19-fd05cecdcb1a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:04,818::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:04,818::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:04,818::task::993::Storage.TaskManager.Task::(_decref) Task=`1757b0f9-3851-41cc-af19-fd05cecdcb1a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:05,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:05,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113871 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:07,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:07,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116056 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:09,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:09,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115368 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:10,015::__init__::225::IOProcess::(_processLogs) (2048) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:10,016::__init__::225::IOProcess::(_processLogs) (2048) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:10,016::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|(2048) Building response >Thread-21::DEBUG::2014-08-22 10:02:10,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:10,017::__init__::225::IOProcess::(_processLogs) (2048) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:10,023::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:02:10,024::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000185582 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:10,024::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:10,025::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) (2049) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) (2049) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) (2049) Building response >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) (2049) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:10,026::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:10,027::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) (2050) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:11,507::__init__::225::IOProcess::(_processLogs) (2050) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:11,508::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|(2050) Building response >Thread-22::DEBUG::2014-08-22 10:02:11,508::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:11,509::__init__::225::IOProcess::(_processLogs) (2050) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:11,515::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:02:11,516::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218358 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:11,516::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:11,517::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:11,517::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:11,517::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:11,517::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:11,518::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:11,518::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:11,518::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:11,518::__init__::225::IOProcess::(_processLogs) (2051) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:11,518::__init__::225::IOProcess::(_processLogs) (2051) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:11,518::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|(2051) Building response >Thread-17::DEBUG::2014-08-22 10:02:11,519::__init__::225::IOProcess::(_processLogs) (2051) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:11,519::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:11,519::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:11,519::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:11,520::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:02:11,520::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:11,520::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:11,520::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) (2052) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) (2052) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) (2052) Building response >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) (2052) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:02:11,521::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:11,522::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:11,522::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:11,522::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:11,522::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:11,522::__init__::225::IOProcess::(_processLogs) (2053) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:11,522::__init__::225::IOProcess::(_processLogs) (2053) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) (2053) Building response >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) (2053) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:02:11,523::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) (2054) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) (2054) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:11,524::__init__::225::IOProcess::(_processLogs) (2054) Building response >Thread-17::DEBUG::2014-08-22 10:02:11,525::__init__::225::IOProcess::(_processLogs) (2054) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:11,525::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:11,525::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:02:11,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:11,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117623 s, 87.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:13,896::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:02:13,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb28b088-0a20-4e99-ac1c-d51360be33f0`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:13,897::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:02:13,897::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:13,897::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb28b088-0a20-4e99-ac1c-d51360be33f0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:13,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb28b088-0a20-4e99-ac1c-d51360be33f0`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:13,897::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:13,898::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:13,898::task::993::Storage.TaskManager.Task::(_decref) Task=`fb28b088-0a20-4e99-ac1c-d51360be33f0`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:02:13,926::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:02:13,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`035c2fde-f61d-4669-baac-40577d48b8e2`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:02:13,926::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:02:13,927::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5dd72dd9-1844-4687-b8ef-adfb9c581bb6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:02:13,927::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:02:13,927::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:02:13,927::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5dd72dd9-1844-4687-b8ef-adfb9c581bb6`::Granted request >Thread-46::DEBUG::2014-08-22 10:02:13,927::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`035c2fde-f61d-4669-baac-40577d48b8e2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:02:13,927::task::993::Storage.TaskManager.Task::(_decref) Task=`035c2fde-f61d-4669-baac-40577d48b8e2`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:02:13,929::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:13,929::task::1191::Storage.TaskManager.Task::(prepare) Task=`035c2fde-f61d-4669-baac-40577d48b8e2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:13,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`035c2fde-f61d-4669-baac-40577d48b8e2`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:02:13,930::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:02:13,930::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:02:13,930::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:02:13,930::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:02:13,930::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:02:13,930::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:02:13,930::task::993::Storage.TaskManager.Task::(_decref) Task=`035c2fde-f61d-4669-baac-40577d48b8e2`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:13,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:14,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114822 s, 89.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:16,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:16,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114929 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:18,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:18,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112545 s, 91.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:20,028::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:20,028::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) (2055) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:20,029::__init__::225::IOProcess::(_processLogs) (2055) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:20,030::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|(2055) Building response >Thread-21::DEBUG::2014-08-22 10:02:20,030::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:20,031::__init__::225::IOProcess::(_processLogs) (2055) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:20,037::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:02:20,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227077 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:20,038::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:20,039::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:20,039::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:20,039::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:20,040::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:20,040::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:20,040::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:20,040::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:20,040::__init__::225::IOProcess::(_processLogs) (2056) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:20,040::__init__::225::IOProcess::(_processLogs) (2056) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:20,041::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|(2056) Building response >Thread-17::DEBUG::2014-08-22 10:02:20,042::__init__::225::IOProcess::(_processLogs) (2056) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:20,042::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:20,042::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:02:20,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dd8d10f-72e2-4564-84c8-0bb27b2b83da`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:20,045::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:02:20,045::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227077', 'lastCheck': '0.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218358', 'lastCheck': '8.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:20,045::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dd8d10f-72e2-4564-84c8-0bb27b2b83da`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227077', 'lastCheck': '0.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218358', 'lastCheck': '8.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:20,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dd8d10f-72e2-4564-84c8-0bb27b2b83da`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:20,045::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:20,045::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:20,046::task::993::Storage.TaskManager.Task::(_decref) Task=`9dd8d10f-72e2-4564-84c8-0bb27b2b83da`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:20,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:20,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115936 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) (2046) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) (2057) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:21,525::__init__::225::IOProcess::(_processLogs) (2057) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:21,526::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2057) Building response >Thread-17::DEBUG::2014-08-22 10:02:21,526::__init__::225::IOProcess::(_processLogs) (2057) Queuing response >Thread-22::DEBUG::2014-08-22 10:02:21,526::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:21,527::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:21,533::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:02:21,534::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000201149 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) (2058) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:21,535::__init__::225::IOProcess::(_processLogs) (2058) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:21,536::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2058) Building response >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) (2058) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:21,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) (2059) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) (2059) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) (2059) Building response >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) (2059) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:21,538::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:21,539::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:21,539::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:02:21,539::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:21,539::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) (2060) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) (2060) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) (2060) Building response >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) (2060) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:21,540::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:21,541::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:02:21,541::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:21,541::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:21,541::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) (2061) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) (2061) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) (2061) Building response >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) (2061) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:21,542::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:02:22,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:22,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120653 s, 84.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:23,975::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:02:23,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3751620-1951-48ff-a9f1-bc8de92e099d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:23,976::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:02:23,976::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:23,976::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3751620-1951-48ff-a9f1-bc8de92e099d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:23,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3751620-1951-48ff-a9f1-bc8de92e099d`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:23,976::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:23,976::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:23,977::task::993::Storage.TaskManager.Task::(_decref) Task=`f3751620-1951-48ff-a9f1-bc8de92e099d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:02:24,018::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:02:24,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`892b2f51-3f55-4ab9-8c71-a722f7e76d3d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:02:24,018::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:02:24,018::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`57ee17eb-4ea4-4281-b4e8-76db3ba1c531`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:02:24,018::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:02:24,019::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:02:24,019::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`57ee17eb-4ea4-4281-b4e8-76db3ba1c531`::Granted request >Thread-46::DEBUG::2014-08-22 10:02:24,019::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`892b2f51-3f55-4ab9-8c71-a722f7e76d3d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:02:24,019::task::993::Storage.TaskManager.Task::(_decref) Task=`892b2f51-3f55-4ab9-8c71-a722f7e76d3d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:02:24,021::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:24,021::task::1191::Storage.TaskManager.Task::(prepare) Task=`892b2f51-3f55-4ab9-8c71-a722f7e76d3d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:24,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`892b2f51-3f55-4ab9-8c71-a722f7e76d3d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:02:24,021::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:02:24,021::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:02:24,021::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:02:24,022::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:02:24,022::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:02:24,022::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:02:24,022::task::993::Storage.TaskManager.Task::(_decref) Task=`892b2f51-3f55-4ab9-8c71-a722f7e76d3d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:24,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:24,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113574 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:26,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:26,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118584 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:28,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:28,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114525 s, 89.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:30,042::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:30,042::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:30,042::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:30,043::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:30,044::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 10:02:30,044::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:30,044::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:30,051::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 10:02:30,051::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000223157 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:30,052::__init__::225::IOProcess::(_processLogs) (2062) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:30,052::__init__::225::IOProcess::(_processLogs) (2062) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:30,052::__init__::225::IOProcess::(_processLogs) (2062) Building response >Thread-17::DEBUG::2014-08-22 10:02:30,052::__init__::225::IOProcess::(_processLogs) (2062) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:30,053::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:30,054::__init__::225::IOProcess::(_processLogs) (2063) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:30,054::__init__::225::IOProcess::(_processLogs) (2063) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:30,054::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2063) Building response >Thread-17::DEBUG::2014-08-22 10:02:30,055::__init__::225::IOProcess::(_processLogs) (2063) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:30,055::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:30,055::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:02:30,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:30,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112391 s, 91.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:31,542::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:31,542::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:31,542::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:31,543::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:31,544::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 10:02:31,544::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:31,544::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:31,551::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 10:02:31,551::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000227469 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:31,552::__init__::225::IOProcess::(_processLogs) (2064) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) (2064) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) (2064) Building response >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) (2064) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:31,553::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) (2065) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:31,554::__init__::225::IOProcess::(_processLogs) (2065) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:31,555::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2065) Building response >Thread-17::DEBUG::2014-08-22 10:02:31,555::__init__::225::IOProcess::(_processLogs) (2065) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:31,555::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:31,556::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:31,556::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:31,556::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:02:31,556::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:31,556::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) (2066) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) (2066) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) (2066) Building response >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) (2066) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:31,557::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:31,558::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:02:31,558::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:31,558::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:31,558::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:31,558::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) (2067) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) (2067) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) (2067) Building response >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) (2067) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:02:31,559::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:31,560::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:31,560::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:31,560::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) (2068) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) (2068) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) (2068) Building response >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) (2068) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:31,561::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:02:32,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:32,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209566 s, 48.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:34,121::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:02:34,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5aaf33b-fee4-42b2-afe9-500758dd5c39`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:34,121::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:02:34,122::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:34,122::task::1191::Storage.TaskManager.Task::(prepare) Task=`c5aaf33b-fee4-42b2-afe9-500758dd5c39`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:34,122::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5aaf33b-fee4-42b2-afe9-500758dd5c39`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:34,122::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:34,122::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:34,122::task::993::Storage.TaskManager.Task::(_decref) Task=`c5aaf33b-fee4-42b2-afe9-500758dd5c39`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:02:34,147::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:02:34,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`346f8849-7143-4678-8779-6c91fc27d331`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:02:34,147::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:02:34,147::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5fbf6982-6b14-4660-8ded-a42b66433902`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:02:34,147::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:02:34,147::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:02:34,147::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5fbf6982-6b14-4660-8ded-a42b66433902`::Granted request >Thread-46::DEBUG::2014-08-22 10:02:34,148::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`346f8849-7143-4678-8779-6c91fc27d331`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:02:34,148::task::993::Storage.TaskManager.Task::(_decref) Task=`346f8849-7143-4678-8779-6c91fc27d331`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:02:34,150::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:34,150::task::1191::Storage.TaskManager.Task::(prepare) Task=`346f8849-7143-4678-8779-6c91fc27d331`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:34,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`346f8849-7143-4678-8779-6c91fc27d331`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:02:34,150::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:02:34,150::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:02:34,150::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:02:34,150::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:02:34,150::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:02:34,150::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:02:34,151::task::993::Storage.TaskManager.Task::(_decref) Task=`346f8849-7143-4678-8779-6c91fc27d331`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:34,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:34,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116447 s, 87.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:35,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`0846edda-d721-4d54-b55c-dfca205e6000`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:35,241::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:02:35,241::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223157', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000227469', 'lastCheck': '3.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:35,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`0846edda-d721-4d54-b55c-dfca205e6000`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223157', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000227469', 'lastCheck': '3.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:35,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`0846edda-d721-4d54-b55c-dfca205e6000`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:35,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:35,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:35,242::task::993::Storage.TaskManager.Task::(_decref) Task=`0846edda-d721-4d54-b55c-dfca205e6000`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:36,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:36,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114206 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:38,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:38,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013448 s, 76.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-08-22 10:02:40,055::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) (2069) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:02:40,057::__init__::225::IOProcess::(_processLogs) (2069) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:02:40,058::__init__::225::IOProcess::(_processLogs) DEBUG|(2069) Building response >Thread-17::DEBUG::2014-08-22 10:02:40,059::__init__::225::IOProcess::(_processLogs) (2069) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:40,059::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,060::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 10:02:40,061::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,061::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 10:02:40,062::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,062::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,063::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,064::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,064::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,065::__init__::225::IOProcess::(_processLogs) (2070) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:02:40,065::__init__::225::IOProcess::(_processLogs) (2070) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:02:40,067::__init__::225::IOProcess::(_processLogs) DEBUG|(2070) Building response >Thread-21::DEBUG::2014-08-22 10:02:40,067::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 10:02:40,068::__init__::225::IOProcess::(_processLogs) (2070) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:40,068::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,068::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 10:02:40,068::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) (2071) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 10:02:40,069::__init__::225::IOProcess::(_processLogs) (2071) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 10:02:40,094::__init__::225::IOProcess::(_processLogs) DEBUG|(2071) Building response >Thread-17::DEBUG::2014-08-22 10:02:40,094::__init__::225::IOProcess::(_processLogs) (2071) Queuing response >Thread-21::DEBUG::2014-08-22 10:02:40,094::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 10:02:40,095::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,095::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:40,095::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,095::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 10:02:40,095::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) (2072) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) (2072) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 10:02:40,096::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|(2072) Building response >Thread-21::DEBUG::2014-08-22 10:02:40,096::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 10:02:40,097::__init__::225::IOProcess::(_processLogs) (2072) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:40,098::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,098::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 10:02:40,098::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,098::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 10:02:40,098::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,098::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) (2073) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) (2073) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) (2073) Building response >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) (2073) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:40,099::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) (2074) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) (2074) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:02:40,100::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|(2074) Building response >Thread-21::DEBUG::2014-08-22 10:02:40,100::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 10:02:40,101::__init__::225::IOProcess::(_processLogs) (2074) Queuing response >Thread-21::INFO::2014-08-22 10:02:40,101::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 10:02:40,101::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 10:02:40,101::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 10:02:40,102::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 10:02:40,102::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,102::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:40,103::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,103::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:02:40,103::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:40,104::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,110::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:02:40,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000220143 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:40,111::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) (2075) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) (2075) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) (2075) Building response >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) (2075) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:40,112::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) (2076) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:40,113::__init__::225::IOProcess::(_processLogs) (2076) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:40,114::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|(2076) Building response >Thread-17::DEBUG::2014-08-22 10:02:40,115::__init__::225::IOProcess::(_processLogs) (2076) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:40,115::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:40,115::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:02:40,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:40,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116515 s, 87.9 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-08-22 10:02:41,561::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 10:02:41,562::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,562::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 10:02:41,562::__init__::225::IOProcess::(_processLogs) (2071) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,562::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,562::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,563::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,563::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,563::__init__::225::IOProcess::(_processLogs) (2077) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:02:41,563::__init__::225::IOProcess::(_processLogs) (2077) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:02:41,563::__init__::225::IOProcess::(_processLogs) (2077) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) (2077) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,564::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,565::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,565::__init__::225::IOProcess::(_processLogs) (2078) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:02:41,565::__init__::225::IOProcess::(_processLogs) (2078) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:02:41,565::__init__::225::IOProcess::(_processLogs) (2078) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,566::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,566::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 10:02:41,568::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 10:02:41,568::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 10:02:41,568::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,569::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,569::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,569::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,569::__init__::225::IOProcess::(_processLogs) (2079) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 10:02:41,569::__init__::225::IOProcess::(_processLogs) (2079) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 10:02:41,585::__init__::225::IOProcess::(_processLogs) (2079) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,585::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-22::DEBUG::2014-08-22 10:02:41,586::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 10:02:41,586::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 10:02:41,586::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,586::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,586::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,587::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,587::__init__::225::IOProcess::(_processLogs) (2080) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 10:02:41,587::__init__::225::IOProcess::(_processLogs) (2080) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 10:02:41,587::__init__::225::IOProcess::(_processLogs) (2080) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,587::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,587::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Sending response sized 832 >Thread-22::DEBUG::2014-08-22 10:02:41,587::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 10:02:41,588::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,588::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 10:02:41,588::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) (2081) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) (2081) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) (2081) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) (2081) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,589::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) (2082) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:02:41,590::__init__::225::IOProcess::(_processLogs) (2082) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:02:41,591::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|(2082) Building response >Thread-22::DEBUG::2014-08-22 10:02:41,591::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 10:02:41,591::__init__::225::IOProcess::(_processLogs) (2082) Queuing response >Thread-22::INFO::2014-08-22 10:02:41,591::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 10:02:41,592::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 10:02:41,592::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 10:02:41,592::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 10:02:41,593::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,593::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:41,593::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,593::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,594::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 10:02:41,594::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:41,594::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,601::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 10:02:41,601::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000206988 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:41,602::__init__::225::IOProcess::(_processLogs) (2083) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:41,602::__init__::225::IOProcess::(_processLogs) (2083) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:41,602::__init__::225::IOProcess::(_processLogs) (2083) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,602::__init__::225::IOProcess::(_processLogs) (2083) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,603::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,604::__init__::225::IOProcess::(_processLogs) (2084) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:41,604::__init__::225::IOProcess::(_processLogs) (2084) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:41,604::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|(2084) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,605::__init__::225::IOProcess::(_processLogs) (2084) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,605::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,605::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:41,605::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,605::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:02:41,605::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,606::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,606::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,606::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,606::__init__::225::IOProcess::(_processLogs) (2085) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:41,606::__init__::225::IOProcess::(_processLogs) (2085) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:41,606::__init__::225::IOProcess::(_processLogs) (2085) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) (2085) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,607::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) (2086) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) (2086) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) (2086) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) (2086) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,608::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:41,609::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:41,609::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:02:41,609::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:41,609::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:41,609::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) (2087) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) (2087) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) (2087) Building response >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) (2087) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:41,610::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:02:42,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:42,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115003 s, 89.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:44,300::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:02:44,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`8156c364-973d-420e-9544-1c1cfd227348`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:44,301::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:02:44,301::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:44,301::task::1191::Storage.TaskManager.Task::(prepare) Task=`8156c364-973d-420e-9544-1c1cfd227348`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:44,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`8156c364-973d-420e-9544-1c1cfd227348`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:44,301::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:44,301::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:44,301::task::993::Storage.TaskManager.Task::(_decref) Task=`8156c364-973d-420e-9544-1c1cfd227348`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:02:44,362::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:02:44,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`841518a5-93b2-49b1-a17d-ad5137e5f8c7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:02:44,363::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:02:44,363::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0660234e-7406-4523-9b5f-c2ef16201e11`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:02:44,363::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:02:44,363::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:02:44,363::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0660234e-7406-4523-9b5f-c2ef16201e11`::Granted request >Thread-46::DEBUG::2014-08-22 10:02:44,363::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`841518a5-93b2-49b1-a17d-ad5137e5f8c7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:02:44,364::task::993::Storage.TaskManager.Task::(_decref) Task=`841518a5-93b2-49b1-a17d-ad5137e5f8c7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:02:44,365::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:44,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`841518a5-93b2-49b1-a17d-ad5137e5f8c7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:44,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`841518a5-93b2-49b1-a17d-ad5137e5f8c7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:02:44,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:02:44,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:02:44,366::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:02:44,366::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:02:44,366::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:02:44,366::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:02:44,366::task::993::Storage.TaskManager.Task::(_decref) Task=`841518a5-93b2-49b1-a17d-ad5137e5f8c7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:44,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:45,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120254 s, 85.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:47,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:47,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117049 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:49,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:49,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115398 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:50,115::__init__::225::IOProcess::(_processLogs) (2080) Queuing responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:50,115::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:50,115::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:50,115::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:50,115::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:50,116::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:50,116::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:50,116::__init__::225::IOProcess::(_processLogs) (2088) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:50,116::__init__::225::IOProcess::(_processLogs) (2088) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:50,116::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2088) Building response >Thread-21::DEBUG::2014-08-22 10:02:50,117::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:50,117::__init__::225::IOProcess::(_processLogs) (2088) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:50,124::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:02:50,125::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000197074 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:50,125::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:50,126::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:50,127::__init__::225::IOProcess::(_processLogs) (2089) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:50,127::__init__::225::IOProcess::(_processLogs) (2089) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:50,127::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2089) Building response >Thread-17::DEBUG::2014-08-22 10:02:50,128::__init__::225::IOProcess::(_processLogs) (2089) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:50,128::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:50,128::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:02:50,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`77e77473-d30c-4639-a746-eea4a5f38382`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:50,411::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:02:50,412::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000197074', 'lastCheck': '0.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206988', 'lastCheck': '8.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:50,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`77e77473-d30c-4639-a746-eea4a5f38382`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000197074', 'lastCheck': '0.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000206988', 'lastCheck': '8.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:02:50,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`77e77473-d30c-4639-a746-eea4a5f38382`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:50,412::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:50,412::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:50,412::task::993::Storage.TaskManager.Task::(_decref) Task=`77e77473-d30c-4639-a746-eea4a5f38382`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:51,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:51,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165756 s, 61.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:51,610::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:51,610::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:51,611::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-22::DEBUG::2014-08-22 10:02:51,612::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:02:51,612::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:51,618::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:51,618::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:51,618::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:51,618::__init__::225::IOProcess::(_processLogs) (2090) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:51,619::__init__::225::IOProcess::(_processLogs) (2090) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:51,619::__init__::225::IOProcess::(_processLogs) (2090) Building response >Thread-17::DEBUG::2014-08-22 10:02:51,619::__init__::225::IOProcess::(_processLogs) (2090) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:51,619::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:02:51,620::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000249265 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:02:51,620::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:51,621::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:51,621::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:02:51,621::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:51,621::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:51,622::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:51,622::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:51,622::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:51,622::__init__::225::IOProcess::(_processLogs) (2091) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:02:51,622::__init__::225::IOProcess::(_processLogs) (2091) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:02:51,622::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2091) Building response >Thread-17::DEBUG::2014-08-22 10:02:51,623::__init__::225::IOProcess::(_processLogs) (2091) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:51,623::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:51,623::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:02:51,623::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:51,624::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:02:51,624::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:51,624::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:51,624::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:51,624::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:51,624::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) (2092) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) (2092) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) (2092) Building response >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) (2092) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:02:51,625::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) (2093) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) (2093) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) (2093) Building response >Thread-17::DEBUG::2014-08-22 10:02:51,626::__init__::225::IOProcess::(_processLogs) (2093) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:51,627::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:51,627::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:02:51,627::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:02:51,627::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:02:51,627::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:02:51,627::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:02:51,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:02:51,628::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:02:51,628::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:02:51,628::__init__::225::IOProcess::(_processLogs) (2094) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:02:51,628::__init__::225::IOProcess::(_processLogs) (2094) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:02:51,628::__init__::225::IOProcess::(_processLogs) (2094) Building response >Thread-17::DEBUG::2014-08-22 10:02:51,629::__init__::225::IOProcess::(_processLogs) (2094) Queuing response >Thread-17::DEBUG::2014-08-22 10:02:51,629::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:02:51,629::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:02:53,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:53,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115365 s, 88.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:02:54,414::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:02:54,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`41acf385-ecbb-4c68-b6b8-eee23754f646`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:02:54,414::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:02:54,414::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:54,415::task::1191::Storage.TaskManager.Task::(prepare) Task=`41acf385-ecbb-4c68-b6b8-eee23754f646`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:02:54,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`41acf385-ecbb-4c68-b6b8-eee23754f646`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:02:54,415::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:02:54,415::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:02:54,415::task::993::Storage.TaskManager.Task::(_decref) Task=`41acf385-ecbb-4c68-b6b8-eee23754f646`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:02:54,472::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:02:54,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bfe15a5-ba58-461a-9867-af7076cb9846`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:02:54,473::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:02:54,473::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2aeb2bb0-3c23-453a-bd9a-17148d67cdfc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:02:54,473::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:02:54,473::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:02:54,473::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2aeb2bb0-3c23-453a-bd9a-17148d67cdfc`::Granted request >Thread-46::DEBUG::2014-08-22 10:02:54,474::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5bfe15a5-ba58-461a-9867-af7076cb9846`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:02:54,474::task::993::Storage.TaskManager.Task::(_decref) Task=`5bfe15a5-ba58-461a-9867-af7076cb9846`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:02:54,476::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:54,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`5bfe15a5-ba58-461a-9867-af7076cb9846`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:02:54,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bfe15a5-ba58-461a-9867-af7076cb9846`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:02:54,476::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:02:54,476::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:02:54,476::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:02:54,476::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:02:54,476::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:02:54,476::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:02:54,476::task::993::Storage.TaskManager.Task::(_decref) Task=`5bfe15a5-ba58-461a-9867-af7076cb9846`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:02:55,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:55,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116642 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:57,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:57,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113631 s, 90.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:02:59,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:02:59,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116143 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:00,128::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) (2095) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:00,129::__init__::225::IOProcess::(_processLogs) (2095) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:00,130::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2095) Building response >Thread-21::DEBUG::2014-08-22 10:03:00,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:00,131::__init__::225::IOProcess::(_processLogs) (2095) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:00,137::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:00,138::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:03:00,138::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000193494 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:00,139::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:00,139::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) (2096) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:00,140::__init__::225::IOProcess::(_processLogs) (2096) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:00,141::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2096) Building response >Thread-17::DEBUG::2014-08-22 10:03:00,142::__init__::225::IOProcess::(_processLogs) (2096) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:00,142::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:00,142::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:03:01,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:01,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114897 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) (2097) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:01,629::__init__::225::IOProcess::(_processLogs) (2097) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:01,630::__init__::225::IOProcess::(_processLogs) (2097) Queuing response >Thread-22::DEBUG::2014-08-22 10:03:01,631::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:01,631::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:01,637::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:03:01,638::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198539 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) (2098) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:01,639::__init__::225::IOProcess::(_processLogs) (2098) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:01,640::__init__::225::IOProcess::(_processLogs) (2098) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:01,641::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:01,641::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:01,641::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:03:01,641::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:01,641::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) (2099) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) (2099) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) (2099) Building response >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) (2099) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:01,642::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) (2100) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:01,643::__init__::225::IOProcess::(_processLogs) (2100) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) (2100) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:01,644::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:01,645::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:01,645::__init__::225::IOProcess::(_processLogs) (2101) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:01,645::__init__::225::IOProcess::(_processLogs) (2101) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:01,645::__init__::225::IOProcess::(_processLogs) (2101) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:01,645::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:01,646::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:03:03,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:03,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116697 s, 87.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:04,541::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:03:04,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c030f84-ca12-46fe-a95e-2f6835f2353f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:04,542::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:03:04,542::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:04,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c030f84-ca12-46fe-a95e-2f6835f2353f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:04,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c030f84-ca12-46fe-a95e-2f6835f2353f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:04,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:04,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:04,543::task::993::Storage.TaskManager.Task::(_decref) Task=`0c030f84-ca12-46fe-a95e-2f6835f2353f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:03:04,566::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:03:04,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e7957e5-919d-4e31-b7b8-d662718cf791`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:03:04,567::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:03:04,567::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6ed81d66-e6bf-4454-8a88-d6a81c6a66d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:03:04,567::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:03:04,567::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:03:04,567::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6ed81d66-e6bf-4454-8a88-d6a81c6a66d2`::Granted request >Thread-46::DEBUG::2014-08-22 10:03:04,568::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7e7957e5-919d-4e31-b7b8-d662718cf791`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:03:04,568::task::993::Storage.TaskManager.Task::(_decref) Task=`7e7957e5-919d-4e31-b7b8-d662718cf791`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:03:04,570::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:04,570::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e7957e5-919d-4e31-b7b8-d662718cf791`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:04,570::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e7957e5-919d-4e31-b7b8-d662718cf791`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:03:04,570::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:03:04,570::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:03:04,570::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:03:04,570::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:03:04,570::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:03:04,570::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:03:04,571::task::993::Storage.TaskManager.Task::(_decref) Task=`7e7957e5-919d-4e31-b7b8-d662718cf791`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:05,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:05,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135985 s, 75.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:05,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`96e2c916-14e1-4707-ae62-74ee2ed30f63`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:05,626::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:03:05,626::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193494', 'lastCheck': '5.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198539', 'lastCheck': '4.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:05,626::task::1191::Storage.TaskManager.Task::(prepare) Task=`96e2c916-14e1-4707-ae62-74ee2ed30f63`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193494', 'lastCheck': '5.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198539', 'lastCheck': '4.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:05,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`96e2c916-14e1-4707-ae62-74ee2ed30f63`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:05,626::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:05,626::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:05,626::task::993::Storage.TaskManager.Task::(_decref) Task=`96e2c916-14e1-4707-ae62-74ee2ed30f63`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:07,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:07,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137254 s, 74.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:09,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:09,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113214 s, 90.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:10,142::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:10,142::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:10,142::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:10,142::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:10,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:10,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:10,143::__init__::225::IOProcess::(_processLogs) (2102) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:10,143::__init__::225::IOProcess::(_processLogs) (2102) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:10,143::__init__::225::IOProcess::(_processLogs) (2102) Queuing response >Thread-21::DEBUG::2014-08-22 10:03:10,144::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:10,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:10,151::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:03:10,152::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000179844 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:10,152::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:10,152::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:10,152::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:10,153::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:10,153::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:10,153::__init__::225::IOProcess::(_processLogs) (2103) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:10,153::__init__::225::IOProcess::(_processLogs) (2103) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:10,153::__init__::225::IOProcess::(_processLogs) (2103) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:10,154::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:10,154::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:03:11,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:11,646::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:11,647::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:11,647::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:11,647::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:11,647::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:11,647::__init__::225::IOProcess::(_processLogs) (2104) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:11,647::__init__::225::IOProcess::(_processLogs) (2104) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:11,648::__init__::225::IOProcess::(_processLogs) (2104) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:11,651::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:03:11,652::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:03:11,659::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000156945 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:11,661::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:11,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:11,664::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:11,665::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:11,666::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:11,667::__init__::225::IOProcess::(_processLogs) (2105) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:11,667::__init__::225::IOProcess::(_processLogs) (2105) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:11,668::__init__::225::IOProcess::(_processLogs) (2105) Building response >Thread-17::DEBUG::2014-08-22 10:03:11,668::__init__::225::IOProcess::(_processLogs) (2105) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:11,669::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:11,669::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:03:11,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307949 s, 33.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:11,694::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:03:11,695::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:11,696::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:11,696::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:11,698::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:11,699::__init__::225::IOProcess::(_processLogs) (2106) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:11,699::__init__::225::IOProcess::(_processLogs) (2106) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:11,700::__init__::225::IOProcess::(_processLogs) (2106) Building response >Thread-17::DEBUG::2014-08-22 10:03:11,700::__init__::225::IOProcess::(_processLogs) (2106) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:11,701::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:11,701::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:11,702::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:03:11,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:11,704::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:11,704::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:11,705::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:11,706::__init__::225::IOProcess::(_processLogs) (2107) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:11,707::__init__::225::IOProcess::(_processLogs) (2107) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:11,707::__init__::225::IOProcess::(_processLogs) (2107) Building response >Thread-17::DEBUG::2014-08-22 10:03:11,708::__init__::225::IOProcess::(_processLogs) (2107) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:11,709::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:11,709::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:11,710::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:03:11,711::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:11,711::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:11,712::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:11,712::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:11,714::__init__::225::IOProcess::(_processLogs) (2108) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:11,715::__init__::225::IOProcess::(_processLogs) (2108) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:11,715::__init__::225::IOProcess::(_processLogs) (2108) Building response >Thread-17::DEBUG::2014-08-22 10:03:11,716::__init__::225::IOProcess::(_processLogs) (2108) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:11,716::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:11,717::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:03:13,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:13,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011948 s, 85.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:14,621::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:03:14,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`67a42f63-d3b2-47da-bef6-6af52b1b2b14`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:14,622::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:03:14,622::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:14,622::task::1191::Storage.TaskManager.Task::(prepare) Task=`67a42f63-d3b2-47da-bef6-6af52b1b2b14`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:14,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`67a42f63-d3b2-47da-bef6-6af52b1b2b14`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:14,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:14,622::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:14,622::task::993::Storage.TaskManager.Task::(_decref) Task=`67a42f63-d3b2-47da-bef6-6af52b1b2b14`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:03:14,685::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:03:14,685::task::595::Storage.TaskManager.Task::(_updateState) Task=`b06a28f0-ac7d-4501-aa18-cef0567ddc68`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:03:14,685::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:03:14,686::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d7171fd5-f689-48a0-9ce9-5fe22e84ab9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:03:14,686::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:03:14,686::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:03:14,686::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`d7171fd5-f689-48a0-9ce9-5fe22e84ab9e`::Granted request >Thread-46::DEBUG::2014-08-22 10:03:14,686::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b06a28f0-ac7d-4501-aa18-cef0567ddc68`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:03:14,686::task::993::Storage.TaskManager.Task::(_decref) Task=`b06a28f0-ac7d-4501-aa18-cef0567ddc68`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:03:14,688::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:14,688::task::1191::Storage.TaskManager.Task::(prepare) Task=`b06a28f0-ac7d-4501-aa18-cef0567ddc68`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:14,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`b06a28f0-ac7d-4501-aa18-cef0567ddc68`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:03:14,688::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:03:14,689::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:03:14,689::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:03:14,689::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:03:14,689::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:03:14,689::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:03:14,689::task::993::Storage.TaskManager.Task::(_decref) Task=`b06a28f0-ac7d-4501-aa18-cef0567ddc68`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:15,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:15,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116753 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:17,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:17,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112893 s, 90.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:19,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:19,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114971 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) (2109) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:20,155::__init__::225::IOProcess::(_processLogs) (2109) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:20,156::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2109) Building response >Thread-17::DEBUG::2014-08-22 10:03:20,157::__init__::225::IOProcess::(_processLogs) (2109) Queuing response >Thread-21::DEBUG::2014-08-22 10:03:20,157::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:20,158::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:20,164::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:03:20,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00020954 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:20,167::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:20,168::__init__::225::IOProcess::(_processLogs) (2110) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:20,168::__init__::225::IOProcess::(_processLogs) (2110) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:20,168::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2110) Building response >Thread-17::DEBUG::2014-08-22 10:03:20,169::__init__::225::IOProcess::(_processLogs) (2110) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:20,169::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:20,169::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:03:20,850::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea7308b1-ad4b-4644-b01a-717f42fff709`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:20,850::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:03:20,850::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00020954', 'lastCheck': '0.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000156945', 'lastCheck': '9.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:20,850::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea7308b1-ad4b-4644-b01a-717f42fff709`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00020954', 'lastCheck': '0.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000156945', 'lastCheck': '9.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:20,850::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea7308b1-ad4b-4644-b01a-717f42fff709`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:20,850::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:20,851::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:20,851::task::993::Storage.TaskManager.Task::(_decref) Task=`ea7308b1-ad4b-4644-b01a-717f42fff709`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:21,715::__init__::225::IOProcess::(_processLogs) (2111) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:21,716::__init__::225::IOProcess::(_processLogs) (2111) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:21,716::__init__::225::IOProcess::(_processLogs) DEBUG|(2111) Building response >Thread-22::DEBUG::2014-08-22 10:03:21,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:21,717::__init__::225::IOProcess::(_processLogs) (2111) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:21,723::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:21,724::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:03:21,724::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000263038 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:21,725::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:21,725::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:21,725::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:21,725::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:21,725::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:21,726::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:21,726::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:21,726::__init__::225::IOProcess::(_processLogs) (2112) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:21,726::__init__::225::IOProcess::(_processLogs) (2112) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:21,726::__init__::225::IOProcess::(_processLogs) (2112) Building response >Thread-17::DEBUG::2014-08-22 10:03:21,727::__init__::225::IOProcess::(_processLogs) (2112) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:21,727::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:21,727::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:21,727::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:03:21,728::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:21,728::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:21,728::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:21,728::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:21,728::__init__::225::IOProcess::(_processLogs) (2113) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:21,728::__init__::225::IOProcess::(_processLogs) (2113) Got request for method 'access' >Dummy-45::DEBUG::2014-08-22 10:03:21,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:21,931::__init__::225::IOProcess::(_processLogs) (2113) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:21,931::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:21,932::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:21,932::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) (2114) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) (2114) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) (2114) Building response >Thread-17::DEBUG::2014-08-22 10:03:21,933::__init__::225::IOProcess::(_processLogs) (2114) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:21,934::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:21,934::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:21,934::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:03:21,934::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:21,934::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) (2115) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) (2115) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) (2115) Building response >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) (2115) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:21,935::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:21,936::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:03:21,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011917 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:23,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:24,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119398 s, 85.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:24,739::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:03:24,739::task::595::Storage.TaskManager.Task::(_updateState) Task=`812a9440-75b8-4301-b18c-41596d9726c1`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:24,739::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:03:24,740::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:24,740::task::1191::Storage.TaskManager.Task::(prepare) Task=`812a9440-75b8-4301-b18c-41596d9726c1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:24,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`812a9440-75b8-4301-b18c-41596d9726c1`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:24,740::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:24,740::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:24,740::task::993::Storage.TaskManager.Task::(_decref) Task=`812a9440-75b8-4301-b18c-41596d9726c1`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:03:24,771::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:03:24,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`87d16c23-2d40-4c8e-8c32-7d1aaa307d4b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:03:24,772::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:03:24,772::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`38c6be04-b13b-46c5-9bd4-3220673e7f7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:03:24,772::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:03:24,772::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:03:24,772::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`38c6be04-b13b-46c5-9bd4-3220673e7f7e`::Granted request >Thread-46::DEBUG::2014-08-22 10:03:24,773::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`87d16c23-2d40-4c8e-8c32-7d1aaa307d4b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:03:24,773::task::993::Storage.TaskManager.Task::(_decref) Task=`87d16c23-2d40-4c8e-8c32-7d1aaa307d4b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:03:24,775::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:24,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`87d16c23-2d40-4c8e-8c32-7d1aaa307d4b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:24,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`87d16c23-2d40-4c8e-8c32-7d1aaa307d4b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:03:24,775::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:03:24,775::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:03:24,775::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:03:24,775::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:03:24,775::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:03:24,775::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:03:24,776::task::993::Storage.TaskManager.Task::(_decref) Task=`87d16c23-2d40-4c8e-8c32-7d1aaa307d4b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:26,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:26,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115879 s, 88.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:28,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:28,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113321 s, 90.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:30,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:30,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124896 s, 82.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:30,174::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:30,175::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:30,176::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:30,176::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:30,177::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:30,178::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:30,178::__init__::225::IOProcess::(_processLogs) (2116) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:30,179::__init__::225::IOProcess::(_processLogs) (2116) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:30,182::__init__::225::IOProcess::(_processLogs) (2116) Queuing response >Thread-21::DEBUG::2014-08-22 10:03:30,183::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:30,183::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:30,190::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:03:30,191::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000219348 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:30,192::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:30,193::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:30,193::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:30,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:30,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:30,197::__init__::225::IOProcess::(_processLogs) (2117) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:30,197::__init__::225::IOProcess::(_processLogs) (2117) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:30,198::__init__::225::IOProcess::(_processLogs) (2117) Building response >Thread-17::DEBUG::2014-08-22 10:03:30,198::__init__::225::IOProcess::(_processLogs) (2117) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:30,198::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:30,199::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:31,935::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) (2118) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) (2118) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) (2118) Building response >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) (2118) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:31,936::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:31,937::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:03:31,937::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:03:31,944::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000190792 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:31,944::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:31,945::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:31,945::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:31,945::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:31,946::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:31,946::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:31,946::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:31,946::__init__::225::IOProcess::(_processLogs) (2119) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:31,946::__init__::225::IOProcess::(_processLogs) (2119) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) (2119) Building response >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) (2119) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:31,947::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) (2120) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) (2120) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) (2120) Building response >Thread-17::DEBUG::2014-08-22 10:03:31,948::__init__::225::IOProcess::(_processLogs) (2120) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:31,949::__init__::225::IOProcess::(_processLogs) (2121) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:31,950::__init__::225::IOProcess::(_processLogs) (2121) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:31,950::__init__::225::IOProcess::(_processLogs) DEBUG|(2121) Building response >Thread-17::DEBUG::2014-08-22 10:03:31,950::__init__::225::IOProcess::(_processLogs) (2121) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:31,950::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:31,950::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:31,951::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:31,951::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:03:31,951::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:31,951::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:31,951::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:31,951::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) (2122) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) (2122) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) (2122) Building response >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) (2122) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:31,952::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:03:32,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:32,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211266 s, 48.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:34,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:34,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117722 s, 87.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:34,818::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:03:34,818::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e3d7467-6220-4421-b9ff-45a622c6bf2f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:34,818::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:03:34,818::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:34,818::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e3d7467-6220-4421-b9ff-45a622c6bf2f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:34,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e3d7467-6220-4421-b9ff-45a622c6bf2f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:34,819::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:34,819::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:34,819::task::993::Storage.TaskManager.Task::(_decref) Task=`5e3d7467-6220-4421-b9ff-45a622c6bf2f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:03:34,860::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:03:34,860::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f600dbf-754b-4cd8-8ed4-70831d1f2702`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:03:34,860::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:03:34,860::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3000555b-58cc-4752-abee-f9b325f29c9d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:03:34,860::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:03:34,861::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:03:34,861::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3000555b-58cc-4752-abee-f9b325f29c9d`::Granted request >Thread-46::DEBUG::2014-08-22 10:03:34,861::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9f600dbf-754b-4cd8-8ed4-70831d1f2702`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:03:34,861::task::993::Storage.TaskManager.Task::(_decref) Task=`9f600dbf-754b-4cd8-8ed4-70831d1f2702`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:03:34,863::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:34,863::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f600dbf-754b-4cd8-8ed4-70831d1f2702`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:34,863::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f600dbf-754b-4cd8-8ed4-70831d1f2702`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:03:34,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:03:34,863::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:03:34,863::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:03:34,864::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:03:34,864::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:03:34,864::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:03:34,864::task::993::Storage.TaskManager.Task::(_decref) Task=`9f600dbf-754b-4cd8-8ed4-70831d1f2702`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 10:03:36,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`502b2e87-2ce1-4767-b214-0f660890acd4`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:36,172::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:03:36,173::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219348', 'lastCheck': '6.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000190792', 'lastCheck': '4.2', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:36,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`502b2e87-2ce1-4767-b214-0f660890acd4`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000219348', 'lastCheck': '6.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000190792', 'lastCheck': '4.2', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:36,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`502b2e87-2ce1-4767-b214-0f660890acd4`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:36,173::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:36,173::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:36,173::task::993::Storage.TaskManager.Task::(_decref) Task=`502b2e87-2ce1-4767-b214-0f660890acd4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:36,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:36,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115369 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:38,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:38,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113121 s, 90.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:40,197::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:40,197::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:40,197::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:40,197::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:40,197::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:40,198::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:40,198::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:40,198::__init__::225::IOProcess::(_processLogs) (2123) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:40,198::__init__::225::IOProcess::(_processLogs) (2123) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:40,198::__init__::225::IOProcess::(_processLogs) (2123) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:40,199::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:03:40,199::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:40,199::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:03:40,206::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000177123 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:40,206::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:40,206::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:40,206::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:40,207::__init__::225::IOProcess::(_processLogs) (2124) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:40,207::__init__::225::IOProcess::(_processLogs) (2124) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:40,207::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:03:40,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:40,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116639 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) (2125) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:41,953::__init__::225::IOProcess::(_processLogs) (2125) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:41,954::__init__::225::IOProcess::(_processLogs) (2125) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:41,954::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:03:41,955::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:41,955::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:03:41,961::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000188438 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:41,962::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:41,962::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:41,962::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:41,962::__init__::225::IOProcess::(_processLogs) (2126) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:41,963::__init__::225::IOProcess::(_processLogs) (2126) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:41,963::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:41,964::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:03:41,964::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:41,964::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) (2127) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) (2127) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) (2127) Building response >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) (2127) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:41,965::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:41,966::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:03:41,966::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:41,966::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:41,966::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:41,966::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) (2128) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) (2128) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) (2128) Building response >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) (2128) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:03:41,967::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:41,968::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:41,968::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:41,968::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:41,969::__init__::225::IOProcess::(_processLogs) (2129) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:41,969::__init__::225::IOProcess::(_processLogs) (2129) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:41,969::__init__::225::IOProcess::(_processLogs) (2129) Building response >Thread-17::DEBUG::2014-08-22 10:03:41,969::__init__::225::IOProcess::(_processLogs) (2129) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:41,969::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:41,969::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:03:42,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:42,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116731 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:44,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:44,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118996 s, 86.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:44,913::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:03:44,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`497f2d18-be77-4fdd-a792-770e0a41174e`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:44,914::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:03:44,914::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:44,914::task::1191::Storage.TaskManager.Task::(prepare) Task=`497f2d18-be77-4fdd-a792-770e0a41174e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:44,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`497f2d18-be77-4fdd-a792-770e0a41174e`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:44,915::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:44,915::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:44,915::task::993::Storage.TaskManager.Task::(_decref) Task=`497f2d18-be77-4fdd-a792-770e0a41174e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:03:44,950::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:03:44,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`63e193a5-3d68-4e25-96ef-582ab4290ffe`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:03:44,951::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:03:44,951::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`03deee1b-b1fe-411c-af8e-8fad566049fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:03:44,951::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:03:44,951::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:03:44,951::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`03deee1b-b1fe-411c-af8e-8fad566049fd`::Granted request >Thread-46::DEBUG::2014-08-22 10:03:44,952::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`63e193a5-3d68-4e25-96ef-582ab4290ffe`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:03:44,952::task::993::Storage.TaskManager.Task::(_decref) Task=`63e193a5-3d68-4e25-96ef-582ab4290ffe`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:03:44,954::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:44,954::task::1191::Storage.TaskManager.Task::(prepare) Task=`63e193a5-3d68-4e25-96ef-582ab4290ffe`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:44,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`63e193a5-3d68-4e25-96ef-582ab4290ffe`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:03:44,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:03:44,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:03:44,954::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:03:44,954::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:03:44,954::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:03:44,954::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:03:44,955::task::993::Storage.TaskManager.Task::(_decref) Task=`63e193a5-3d68-4e25-96ef-582ab4290ffe`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:46,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:46,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115192 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:48,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:48,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114319 s, 89.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:50,209::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:50,209::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:50,209::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:50,209::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:50,209::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:50,209::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:50,210::__init__::225::IOProcess::(_processLogs) (2130) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:50,210::__init__::225::IOProcess::(_processLogs) (2130) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:50,210::__init__::225::IOProcess::(_processLogs) DEBUG|(2130) Building response >Thread-17::DEBUG::2014-08-22 10:03:50,211::__init__::225::IOProcess::(_processLogs) (2130) Queuing response >Thread-21::DEBUG::2014-08-22 10:03:50,211::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:50,212::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:50,218::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:03:50,219::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000198636 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:50,220::__init__::225::IOProcess::(_processLogs) (2131) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:50,221::__init__::225::IOProcess::(_processLogs) (2131) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:50,221::__init__::225::IOProcess::(_processLogs) (2131) Building response >Thread-17::DEBUG::2014-08-22 10:03:50,221::__init__::225::IOProcess::(_processLogs) (2131) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:50,221::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:03:50,222::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:03:50,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:50,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117617 s, 87.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:51,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`51ca64cc-84cc-4821-86da-acfdf25dd3ac`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:51,413::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:03:51,413::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000198636', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000188438', 'lastCheck': '9.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:51,413::task::1191::Storage.TaskManager.Task::(prepare) Task=`51ca64cc-84cc-4821-86da-acfdf25dd3ac`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000198636', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000188438', 'lastCheck': '9.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:03:51,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`51ca64cc-84cc-4821-86da-acfdf25dd3ac`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:51,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:51,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:51,414::task::993::Storage.TaskManager.Task::(_decref) Task=`51ca64cc-84cc-4821-86da-acfdf25dd3ac`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:03:51,969::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:51,969::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:51,970::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:51,971::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-22::DEBUG::2014-08-22 10:03:51,971::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:03:51,971::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:51,977::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:03:51,978::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000212187 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:03:51,979::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:51,979::__init__::225::IOProcess::(_processLogs) (2132) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:51,979::__init__::225::IOProcess::(_processLogs) (2132) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:51,979::__init__::225::IOProcess::(_processLogs) (2132) Building response >Thread-17::DEBUG::2014-08-22 10:03:51,979::__init__::225::IOProcess::(_processLogs) (2132) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:51,979::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) (2133) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:03:51,980::__init__::225::IOProcess::(_processLogs) (2133) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:03:51,981::__init__::225::IOProcess::(_processLogs) DEBUG|(2133) Building response >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) (2133) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:51,982::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:51,983::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:51,983::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:51,983::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:51,983::__init__::225::IOProcess::(_processLogs) (2134) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) (2134) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) (2134) Building response >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) (2134) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:03:51,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) (2135) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) (2135) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) (2135) Building response >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) (2135) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:51,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:51,986::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:03:51,986::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:03:51,986::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:03:51,986::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:03:51,986::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:03:51,986::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:03:51,987::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:03:51,987::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:03:51,987::__init__::225::IOProcess::(_processLogs) (2136) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:03:51,987::__init__::225::IOProcess::(_processLogs) (2136) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:03:51,987::__init__::225::IOProcess::(_processLogs) (2136) Building response >Thread-17::DEBUG::2014-08-22 10:03:51,987::__init__::225::IOProcess::(_processLogs) (2136) Queuing response >Thread-17::DEBUG::2014-08-22 10:03:51,988::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:03:51,988::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:03:52,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:52,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129668 s, 79.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:54,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:54,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113694 s, 90.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:03:55,026::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-18::DEBUG::2014-08-22 10:03:55,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`71f56853-dc8e-4673-a0dc-73e609a90bdf`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:03:55,026::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:03:55,027::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:55,027::task::1191::Storage.TaskManager.Task::(prepare) Task=`71f56853-dc8e-4673-a0dc-73e609a90bdf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:03:55,027::task::595::Storage.TaskManager.Task::(_updateState) Task=`71f56853-dc8e-4673-a0dc-73e609a90bdf`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:03:55,027::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:03:55,027::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:03:55,027::task::993::Storage.TaskManager.Task::(_decref) Task=`71f56853-dc8e-4673-a0dc-73e609a90bdf`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:03:55,070::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-46::DEBUG::2014-08-22 10:03:55,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6d72771-1f8e-4c57-b23c-2ef1cfab68f8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:03:55,070::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:03:55,070::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3783b777-a28f-409c-aac2-6ab55f3092a5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:03:55,070::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:03:55,071::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:03:55,071::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3783b777-a28f-409c-aac2-6ab55f3092a5`::Granted request >Thread-46::DEBUG::2014-08-22 10:03:55,071::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6d72771-1f8e-4c57-b23c-2ef1cfab68f8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:03:55,071::task::993::Storage.TaskManager.Task::(_decref) Task=`b6d72771-1f8e-4c57-b23c-2ef1cfab68f8`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:03:55,073::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:55,073::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6d72771-1f8e-4c57-b23c-2ef1cfab68f8`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:03:55,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6d72771-1f8e-4c57-b23c-2ef1cfab68f8`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:03:55,073::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:03:55,073::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:03:55,073::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:03:55,074::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:03:55,074::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:03:55,074::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:03:55,074::task::993::Storage.TaskManager.Task::(_decref) Task=`b6d72771-1f8e-4c57-b23c-2ef1cfab68f8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:03:56,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:56,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113489 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:03:58,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:03:59,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111373 s, 91.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) (2137) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:00,223::__init__::225::IOProcess::(_processLogs) (2137) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:00,224::__init__::225::IOProcess::(_processLogs) DEBUG|(2137) Building response >Thread-21::DEBUG::2014-08-22 10:04:00,225::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:00,225::__init__::225::IOProcess::(_processLogs) (2137) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:00,231::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:04:00,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000291112 s, 1.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:00,232::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:00,233::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) (2138) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:00,234::__init__::225::IOProcess::(_processLogs) (2138) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:00,235::__init__::225::IOProcess::(_processLogs) DEBUG|(2138) Building response >Thread-17::DEBUG::2014-08-22 10:04:00,236::__init__::225::IOProcess::(_processLogs) (2138) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:00,236::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:00,236::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:04:01,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:01,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115278 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:01,988::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:01,989::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:01,990::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:01,990::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:01,991::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:01,992::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:01,992::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:01,993::__init__::225::IOProcess::(_processLogs) (2139) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:01,994::__init__::225::IOProcess::(_processLogs) (2139) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:01,996::__init__::225::IOProcess::(_processLogs) DEBUG|(2139) Building response >Thread-22::DEBUG::2014-08-22 10:04:01,997::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:01,998::__init__::225::IOProcess::(_processLogs) (2139) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:02,004::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:02,006::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:04:02,006::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000234376 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:02,008::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:02,009::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:02,010::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:02,010::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:02,011::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:02,011::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:02,014::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:02,016::__init__::225::IOProcess::(_processLogs) (2140) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:02,016::__init__::225::IOProcess::(_processLogs) (2140) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:02,017::__init__::225::IOProcess::(_processLogs) (2140) Building response >Thread-17::DEBUG::2014-08-22 10:04:02,017::__init__::225::IOProcess::(_processLogs) (2140) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:02,018::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:02,019::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:02,020::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:02,021::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:04:02,021::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:02,022::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:02,022::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:02,023::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:02,024::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:02,025::__init__::225::IOProcess::(_processLogs) DEBUG|(2141) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:02,026::__init__::225::IOProcess::(_processLogs) (2141) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:02,027::__init__::225::IOProcess::(_processLogs) (2141) Building response >Thread-17::DEBUG::2014-08-22 10:04:02,027::__init__::225::IOProcess::(_processLogs) (2141) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:02,028::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:02,028::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:02,030::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:02,030::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:04:02,032::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:02,033::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:02,033::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:02,034::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:02,035::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:02,035::__init__::225::IOProcess::(_processLogs) (2142) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:02,036::__init__::225::IOProcess::(_processLogs) (2142) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:02,037::__init__::225::IOProcess::(_processLogs) (2142) Building response >Thread-17::DEBUG::2014-08-22 10:04:02,037::__init__::225::IOProcess::(_processLogs) (2142) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:02,038::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:02,038::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:02,040::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:02,040::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:04:02,041::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:02,042::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:02,043::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:02,043::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:02,044::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:02,045::__init__::225::IOProcess::(_processLogs) (2143) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:02,045::__init__::225::IOProcess::(_processLogs) (2143) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:02,047::__init__::225::IOProcess::(_processLogs) DEBUG|(2143) Building response >Thread-17::DEBUG::2014-08-22 10:04:02,048::__init__::225::IOProcess::(_processLogs) (2143) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:02,049::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:02,049::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:04:03,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:03,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118018 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:05,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-18::DEBUG::2014-08-22 10:04:05,158::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:04:05,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`9976386b-2d3c-467a-a084-a736c65421d3`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:05,159::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:04:05,159::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:05,159::task::1191::Storage.TaskManager.Task::(prepare) Task=`9976386b-2d3c-467a-a084-a736c65421d3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:05,159::task::595::Storage.TaskManager.Task::(_updateState) Task=`9976386b-2d3c-467a-a084-a736c65421d3`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:05,159::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:05,159::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:05,159::task::993::Storage.TaskManager.Task::(_decref) Task=`9976386b-2d3c-467a-a084-a736c65421d3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:05,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115013 s, 89.0 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 10:04:05,196::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:04:05,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`a31b59a0-728f-4b53-8d4e-6795db260ea9`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:04:05,196::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:04:05,196::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`63ef760d-c544-424b-a786-37937d93820e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:04:05,196::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:04:05,196::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:04:05,197::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`63ef760d-c544-424b-a786-37937d93820e`::Granted request >Thread-46::DEBUG::2014-08-22 10:04:05,197::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a31b59a0-728f-4b53-8d4e-6795db260ea9`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:04:05,197::task::993::Storage.TaskManager.Task::(_decref) Task=`a31b59a0-728f-4b53-8d4e-6795db260ea9`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:04:05,199::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:05,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`a31b59a0-728f-4b53-8d4e-6795db260ea9`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:05,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`a31b59a0-728f-4b53-8d4e-6795db260ea9`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:04:05,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:04:05,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:04:05,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:04:05,199::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:04:05,200::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:04:05,200::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:04:05,200::task::993::Storage.TaskManager.Task::(_decref) Task=`a31b59a0-728f-4b53-8d4e-6795db260ea9`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 10:04:06,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`509a77aa-51f0-41a7-ae53-36a5762f1e48`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:06,593::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:04:06,593::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000291112', 'lastCheck': '6.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234376', 'lastCheck': '4.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:06,593::task::1191::Storage.TaskManager.Task::(prepare) Task=`509a77aa-51f0-41a7-ae53-36a5762f1e48`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000291112', 'lastCheck': '6.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000234376', 'lastCheck': '4.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:06,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`509a77aa-51f0-41a7-ae53-36a5762f1e48`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:06,594::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:06,594::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:06,594::task::993::Storage.TaskManager.Task::(_decref) Task=`509a77aa-51f0-41a7-ae53-36a5762f1e48`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:07,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:07,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113476 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:09,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:09,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112932 s, 90.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:10,236::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) (2144) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:10,237::__init__::225::IOProcess::(_processLogs) (2144) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:10,238::__init__::225::IOProcess::(_processLogs) DEBUG|(2144) Building response >Thread-21::DEBUG::2014-08-22 10:04:10,238::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:10,239::__init__::225::IOProcess::(_processLogs) (2144) Queuing response >Thread-21::DEBUG::2014-08-22 10:04:10,246::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000178682 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:10,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:10,246::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:10,246::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:10,246::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:10,246::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:10,247::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:10,247::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:10,247::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:10,247::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:10,247::__init__::225::IOProcess::(_processLogs) (2145) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:10,247::__init__::225::IOProcess::(_processLogs) (2145) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:10,248::__init__::225::IOProcess::(_processLogs) DEBUG|(2145) Building response >Thread-17::DEBUG::2014-08-22 10:04:10,249::__init__::225::IOProcess::(_processLogs) (2145) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:10,249::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:10,249::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:04:11,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:11,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162207 s, 63.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:12,048::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:12,048::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:12,048::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) (2146) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) (2146) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:12,049::__init__::225::IOProcess::(_processLogs) DEBUG|(2146) Building response >Thread-17::DEBUG::2014-08-22 10:04:12,050::__init__::225::IOProcess::(_processLogs) (2146) Queuing response >Thread-22::DEBUG::2014-08-22 10:04:12,050::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:12,050::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:12,057::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:04:12,058::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000187371 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:12,058::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:12,058::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) (2147) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) (2147) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:12,059::__init__::225::IOProcess::(_processLogs) (2147) Building response >Thread-17::DEBUG::2014-08-22 10:04:12,060::__init__::225::IOProcess::(_processLogs) (2147) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:12,061::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:12,061::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:12,061::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:12,061::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:04:12,061::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:12,061::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) (2148) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) (2148) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) (2148) Building response >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) (2148) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:12,062::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:12,063::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:12,063::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:04:12,063::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:12,063::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:12,063::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) (2149) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) (2149) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) (2149) Building response >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) (2149) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:12,064::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:04:12,065::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:12,065::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:12,065::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) (2150) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) (2150) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) (2150) Building response >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) (2150) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:12,066::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:04:13,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:13,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112067 s, 91.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:15,249::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:04:15,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`5567a627-97ca-4897-bb2d-3fd1cb529cef`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:15,250::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:04:15,250::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:15,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`5567a627-97ca-4897-bb2d-3fd1cb529cef`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:15,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`5567a627-97ca-4897-bb2d-3fd1cb529cef`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:15,250::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:15,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:15,251::task::993::Storage.TaskManager.Task::(_decref) Task=`5567a627-97ca-4897-bb2d-3fd1cb529cef`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:04:15,298::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:04:15,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`37627398-e13a-4fd0-aa74-fea2f4863eb0`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:04:15,299::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:04:15,299::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9e8edaca-4aa6-47ee-9e4d-2d4e9b062c87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:04:15,299::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:04:15,299::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:04:15,299::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9e8edaca-4aa6-47ee-9e4d-2d4e9b062c87`::Granted request >Thread-46::DEBUG::2014-08-22 10:04:15,299::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`37627398-e13a-4fd0-aa74-fea2f4863eb0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:04:15,299::task::993::Storage.TaskManager.Task::(_decref) Task=`37627398-e13a-4fd0-aa74-fea2f4863eb0`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:04:15,301::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:15,301::task::1191::Storage.TaskManager.Task::(prepare) Task=`37627398-e13a-4fd0-aa74-fea2f4863eb0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:15,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`37627398-e13a-4fd0-aa74-fea2f4863eb0`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:04:15,302::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:04:15,302::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:04:15,302::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:04:15,302::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:04:15,302::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:04:15,302::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:04:15,302::task::993::Storage.TaskManager.Task::(_decref) Task=`37627398-e13a-4fd0-aa74-fea2f4863eb0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:15,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:15,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114361 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:17,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:17,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113867 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:19,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:19,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114095 s, 89.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:20,249::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:20,249::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) (2151) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) (2151) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) (2151) Building response >Thread-17::DEBUG::2014-08-22 10:04:20,250::__init__::225::IOProcess::(_processLogs) (2151) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:20,251::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 10:04:20,252::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:20,252::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:04:20,259::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000235915 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:20,259::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:20,259::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:20,259::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:20,260::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:20,260::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:20,260::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:20,260::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:20,261::__init__::225::IOProcess::(_processLogs) (2152) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:20,261::__init__::225::IOProcess::(_processLogs) (2152) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:20,261::__init__::225::IOProcess::(_processLogs) (2152) Building response >Thread-17::DEBUG::2014-08-22 10:04:20,262::__init__::225::IOProcess::(_processLogs) (2152) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:20,262::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:20,262::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:04:21,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:21,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117886 s, 86.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:21,792::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd798f2c-c956-406b-afc4-1a26cb2e6417`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:21,792::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:04:21,793::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000235915', 'lastCheck': '1.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000187371', 'lastCheck': '9.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:21,793::task::1191::Storage.TaskManager.Task::(prepare) Task=`fd798f2c-c956-406b-afc4-1a26cb2e6417`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000235915', 'lastCheck': '1.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000187371', 'lastCheck': '9.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:21,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd798f2c-c956-406b-afc4-1a26cb2e6417`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:21,793::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:21,793::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:21,793::task::993::Storage.TaskManager.Task::(_decref) Task=`fd798f2c-c956-406b-afc4-1a26cb2e6417`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:04:22,066::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:22,066::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:22,066::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:22,066::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:22,066::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:22,066::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:22,067::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:22,067::__init__::225::IOProcess::(_processLogs) (2153) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:22,067::__init__::225::IOProcess::(_processLogs) (2153) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:22,067::__init__::225::IOProcess::(_processLogs) (2153) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:22,068::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:04:22,068::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:22,068::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:04:22,075::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000307414 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:22,075::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:22,075::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:22,075::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:22,076::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:22,076::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:22,076::__init__::225::IOProcess::(_processLogs) (2154) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:22,076::__init__::225::IOProcess::(_processLogs) (2154) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:22,076::__init__::225::IOProcess::(_processLogs) (2154) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:22,077::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:22,077::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:22,077::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) (2155) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) (2155) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:22,078::__init__::225::IOProcess::(_processLogs) (2155) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:22,079::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:22,079::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:22,079::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:04:22,079::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) (2156) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) (2156) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) (2156) Building response >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) (2156) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:22,080::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:22,081::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:04:22,081::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:22,081::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:22,081::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) (2157) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) (2157) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) (2157) Building response >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) (2157) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:22,082::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:04:23,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:23,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011976 s, 85.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:25,353::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:04:25,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`62e0c4d6-64fd-4123-9159-e059c9baf23c`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:25,354::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:04:25,354::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:25,354::task::1191::Storage.TaskManager.Task::(prepare) Task=`62e0c4d6-64fd-4123-9159-e059c9baf23c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:25,354::task::595::Storage.TaskManager.Task::(_updateState) Task=`62e0c4d6-64fd-4123-9159-e059c9baf23c`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:25,354::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:25,354::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:25,354::task::993::Storage.TaskManager.Task::(_decref) Task=`62e0c4d6-64fd-4123-9159-e059c9baf23c`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:04:25,378::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:04:25,378::task::595::Storage.TaskManager.Task::(_updateState) Task=`39f2e459-0bb4-4858-9ed7-83241c1a9038`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:04:25,378::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:04:25,378::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2e5f6f16-7b9d-4087-96d2-f7d52fc15920`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:04:25,378::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:04:25,378::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:04:25,378::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2e5f6f16-7b9d-4087-96d2-f7d52fc15920`::Granted request >Thread-46::DEBUG::2014-08-22 10:04:25,379::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`39f2e459-0bb4-4858-9ed7-83241c1a9038`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:04:25,379::task::993::Storage.TaskManager.Task::(_decref) Task=`39f2e459-0bb4-4858-9ed7-83241c1a9038`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:04:25,381::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:25,381::task::1191::Storage.TaskManager.Task::(prepare) Task=`39f2e459-0bb4-4858-9ed7-83241c1a9038`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:25,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`39f2e459-0bb4-4858-9ed7-83241c1a9038`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:04:25,381::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:04:25,381::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:04:25,381::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:04:25,381::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:04:25,381::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:04:25,381::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:04:25,382::task::993::Storage.TaskManager.Task::(_decref) Task=`39f2e459-0bb4-4858-9ed7-83241c1a9038`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:25,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:25,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114402 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:27,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:27,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115007 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:29,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:29,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115342 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) (2158) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:30,262::__init__::225::IOProcess::(_processLogs) (2158) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:30,263::__init__::225::IOProcess::(_processLogs) (2158) Queuing response >Thread-21::DEBUG::2014-08-22 10:04:30,264::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:30,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:30,270::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:04:30,271::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000222798 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:30,272::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:30,272::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:30,272::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:30,272::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:30,272::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:30,272::__init__::225::IOProcess::(_processLogs) (2159) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:30,273::__init__::225::IOProcess::(_processLogs) (2159) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:30,273::__init__::225::IOProcess::(_processLogs) (2159) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:30,274::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:30,274::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:04:31,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:31,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119031 s, 86.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:32,083::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:32,083::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:32,084::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:32,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:32,087::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 10:04:32,088::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:32,088::__init__::225::IOProcess::(_processLogs) (2160) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:32,095::__init__::225::IOProcess::(_processLogs) (2160) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:32,096::__init__::225::IOProcess::(_processLogs) (2160) Building response >Thread-22::DEBUG::2014-08-22 10:04:32,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000166448 s, 3.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:32,098::__init__::225::IOProcess::(_processLogs) (2160) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:32,099::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:32,099::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:32,100::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:32,101::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:32,101::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:32,103::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:32,104::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:32,105::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:32,105::__init__::225::IOProcess::(_processLogs) (2161) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:32,106::__init__::225::IOProcess::(_processLogs) (2161) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:32,106::__init__::225::IOProcess::(_processLogs) (2161) Building response >Thread-17::DEBUG::2014-08-22 10:04:32,107::__init__::225::IOProcess::(_processLogs) (2161) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:32,107::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:32,108::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:32,108::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:32,109::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:04:32,109::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:32,110::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:32,110::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:32,111::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:32,112::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:32,113::__init__::225::IOProcess::(_processLogs) (2162) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:32,113::__init__::225::IOProcess::(_processLogs) (2162) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:32,113::__init__::225::IOProcess::(_processLogs) (2162) Building response >Thread-17::DEBUG::2014-08-22 10:04:32,114::__init__::225::IOProcess::(_processLogs) (2162) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:32,114::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:32,115::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:32,116::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:32,116::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:04:32,117::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:32,117::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:32,118::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:32,118::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:32,119::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:32,120::__init__::225::IOProcess::(_processLogs) (2163) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:32,120::__init__::225::IOProcess::(_processLogs) (2163) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:32,121::__init__::225::IOProcess::(_processLogs) (2163) Building response >Thread-17::DEBUG::2014-08-22 10:04:32,121::__init__::225::IOProcess::(_processLogs) (2163) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:32,122::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:32,122::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:32,123::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:32,123::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:04:32,124::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:32,124::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:32,125::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:32,125::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:32,126::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:32,127::__init__::225::IOProcess::(_processLogs) (2164) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:32,128::__init__::225::IOProcess::(_processLogs) (2164) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:32,128::__init__::225::IOProcess::(_processLogs) (2164) Building response >Thread-17::DEBUG::2014-08-22 10:04:32,129::__init__::225::IOProcess::(_processLogs) (2164) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:32,129::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:32,130::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:04:33,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:33,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112873 s, 90.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:35,422::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:04:35,422::task::595::Storage.TaskManager.Task::(_updateState) Task=`da8d1734-7b74-4cf6-a404-29713aedc6c5`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:35,422::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:04:35,423::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:35,423::task::1191::Storage.TaskManager.Task::(prepare) Task=`da8d1734-7b74-4cf6-a404-29713aedc6c5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:35,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`da8d1734-7b74-4cf6-a404-29713aedc6c5`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:35,423::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:35,423::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:35,423::task::993::Storage.TaskManager.Task::(_decref) Task=`da8d1734-7b74-4cf6-a404-29713aedc6c5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:04:35,465::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:04:35,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e13e8a9-3e8f-434a-8749-8c44a5e93747`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:04:35,465::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:04:35,465::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b0c4add0-62e2-4651-ad92-ccf14e9788f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:04:35,465::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:04:35,466::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:04:35,466::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b0c4add0-62e2-4651-ad92-ccf14e9788f5`::Granted request >Thread-46::DEBUG::2014-08-22 10:04:35,467::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7e13e8a9-3e8f-434a-8749-8c44a5e93747`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:04:35,467::task::993::Storage.TaskManager.Task::(_decref) Task=`7e13e8a9-3e8f-434a-8749-8c44a5e93747`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:04:35,469::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:35,469::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e13e8a9-3e8f-434a-8749-8c44a5e93747`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:35,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e13e8a9-3e8f-434a-8749-8c44a5e93747`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:04:35,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:04:35,469::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:04:35,469::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:04:35,469::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:04:35,469::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:04:35,469::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:04:35,470::task::993::Storage.TaskManager.Task::(_decref) Task=`7e13e8a9-3e8f-434a-8749-8c44a5e93747`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:35,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:35,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116226 s, 88.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:37,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b017258-26b8-48cc-b1f4-acf79dcb1093`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:37,010::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:04:37,011::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000222798', 'lastCheck': '6.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000166448', 'lastCheck': '4.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:37,011::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b017258-26b8-48cc-b1f4-acf79dcb1093`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000222798', 'lastCheck': '6.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000166448', 'lastCheck': '4.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:37,011::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b017258-26b8-48cc-b1f4-acf79dcb1093`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:37,011::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:37,011::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:37,011::task::993::Storage.TaskManager.Task::(_decref) Task=`8b017258-26b8-48cc-b1f4-acf79dcb1093`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:37,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:38,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114561 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:40,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:40,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114358 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) (2165) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) (2165) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:40,275::__init__::225::IOProcess::(_processLogs) (2165) Building response >Thread-17::DEBUG::2014-08-22 10:04:40,276::__init__::225::IOProcess::(_processLogs) DEBUG|(2165) Queuing response >Thread-21::DEBUG::2014-08-22 10:04:40,277::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:40,277::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:40,284::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:04:40,284::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000208209 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:40,285::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:40,285::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:40,285::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:40,285::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:40,285::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:40,285::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:40,286::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:40,286::__init__::225::IOProcess::(_processLogs) (2166) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:40,286::__init__::225::IOProcess::(_processLogs) (2166) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:40,286::__init__::225::IOProcess::(_processLogs) DEBUG|(2166) Building response >Thread-17::DEBUG::2014-08-22 10:04:40,287::__init__::225::IOProcess::(_processLogs) (2166) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:40,287::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:40,287::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:04:42,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:42,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118347 s, 86.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:42,128::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:42,129::__init__::225::IOProcess::(_processLogs) (2167) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:42,129::__init__::225::IOProcess::(_processLogs) (2167) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:42,129::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2167) Building response >Thread-17::DEBUG::2014-08-22 10:04:42,130::__init__::225::IOProcess::(_processLogs) (2167) Queuing response >Thread-22::DEBUG::2014-08-22 10:04:42,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:42,130::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:42,137::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:04:42,137::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218419 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:42,138::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:42,138::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:42,138::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) (2168) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) (2168) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) (2168) Building response >Thread-17::DEBUG::2014-08-22 10:04:42,139::__init__::225::IOProcess::(_processLogs) (2168) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:42,140::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:04:42,140::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:42,141::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:42,141::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:04:42,141::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:42,141::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:42,141::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) (2169) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) (2169) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) (2169) Building response >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) (2169) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:04:42,142::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:42,143::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:42,143::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:42,143::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:42,143::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:42,143::__init__::225::IOProcess::(_processLogs) (2170) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:42,143::__init__::225::IOProcess::(_processLogs) (2170) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) (2170) Building response >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) (2170) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:04:42,144::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:42,145::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:42,145::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:42,145::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:42,145::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:42,145::__init__::225::IOProcess::(_processLogs) (2171) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:42,145::__init__::225::IOProcess::(_processLogs) (2171) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:42,146::__init__::225::IOProcess::(_processLogs) (2171) Building response >Thread-17::DEBUG::2014-08-22 10:04:42,146::__init__::225::IOProcess::(_processLogs) (2171) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:42,146::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:42,146::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:04:44,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:44,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118404 s, 86.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:45,538::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:04:45,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac1a6291-6e77-472b-8da4-9754a5b73b15`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:45,539::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:04:45,539::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:45,539::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac1a6291-6e77-472b-8da4-9754a5b73b15`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:45,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac1a6291-6e77-472b-8da4-9754a5b73b15`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:45,539::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:45,539::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:45,539::task::993::Storage.TaskManager.Task::(_decref) Task=`ac1a6291-6e77-472b-8da4-9754a5b73b15`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:04:45,569::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:04:45,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c4a75ba-0606-4ce3-8def-80a819fa88c7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:04:45,569::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:04:45,569::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12d6f8d1-4330-458d-bc36-d7e63f98870c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:04:45,570::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:04:45,570::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:04:45,570::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`12d6f8d1-4330-458d-bc36-d7e63f98870c`::Granted request >Thread-46::DEBUG::2014-08-22 10:04:45,570::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0c4a75ba-0606-4ce3-8def-80a819fa88c7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:04:45,570::task::993::Storage.TaskManager.Task::(_decref) Task=`0c4a75ba-0606-4ce3-8def-80a819fa88c7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:04:45,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:45,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c4a75ba-0606-4ce3-8def-80a819fa88c7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:45,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c4a75ba-0606-4ce3-8def-80a819fa88c7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:04:45,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:04:45,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:04:45,572::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:04:45,573::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:04:45,573::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:04:45,573::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:04:45,573::task::993::Storage.TaskManager.Task::(_decref) Task=`0c4a75ba-0606-4ce3-8def-80a819fa88c7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:46,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:46,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115351 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:48,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:48,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116985 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:50,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:50,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115181 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:50,308::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:50,309::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:50,310::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:50,310::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:50,311::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:50,312::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:50,312::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:50,313::__init__::225::IOProcess::(_processLogs) (2172) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:50,313::__init__::225::IOProcess::(_processLogs) (2172) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:50,316::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2172) Building response >Thread-21::DEBUG::2014-08-22 10:04:50,317::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:50,317::__init__::225::IOProcess::(_processLogs) (2172) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:50,324::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:50,325::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:04:50,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000174552 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:50,327::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:50,328::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:04:50,329::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:50,329::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:50,330::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:50,330::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:50,333::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:50,334::__init__::225::IOProcess::(_processLogs) (2173) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:50,335::__init__::225::IOProcess::(_processLogs) (2173) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:50,336::__init__::225::IOProcess::(_processLogs) (2173) Building response >Thread-17::DEBUG::2014-08-22 10:04:50,336::__init__::225::IOProcess::(_processLogs) (2173) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:50,336::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:50,336::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) (2174) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:52,146::__init__::225::IOProcess::(_processLogs) (2174) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:52,147::__init__::225::IOProcess::(_processLogs) DEBUG|(2174) Building response >Thread-17::DEBUG::2014-08-22 10:04:52,147::__init__::225::IOProcess::(_processLogs) (2174) Queuing response >Thread-22::DEBUG::2014-08-22 10:04:52,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:04:52,148::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:04:52,154::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000182653 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:04:52,154::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:52,155::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:52,156::__init__::225::IOProcess::(_processLogs) (2175) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:04:52,156::__init__::225::IOProcess::(_processLogs) (2175) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:04:52,156::__init__::225::IOProcess::(_processLogs) (2175) Building response >Thread-17::DEBUG::2014-08-22 10:04:52,156::__init__::225::IOProcess::(_processLogs) DEBUG|(2175) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:52,157::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:52,157::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:04:52,157::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:52,158::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:04:52,158::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:52,158::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:52,158::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) (2176) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) (2176) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) (2176) Building response >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) (2176) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:04:52,159::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:52,160::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:52,160::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:52,160::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:52,160::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) (2177) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) (2177) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) (2177) Building response >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) (2177) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:04:52,161::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:04:52,162::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:04:52,162::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:04:52,162::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) (2178) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) (2178) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) (2178) Building response >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) (2178) Queuing response >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:04:52,163::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::DEBUG::2014-08-22 10:04:52,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b66dff-811c-4f7e-8ee4-084c5095d9b2`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:52,205::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:04:52,205::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000174552', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182653', 'lastCheck': '0.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:52,205::task::1191::Storage.TaskManager.Task::(prepare) Task=`68b66dff-811c-4f7e-8ee4-084c5095d9b2`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000174552', 'lastCheck': '1.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000182653', 'lastCheck': '0.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:04:52,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b66dff-811c-4f7e-8ee4-084c5095d9b2`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:52,205::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:52,206::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:52,206::task::993::Storage.TaskManager.Task::(_decref) Task=`68b66dff-811c-4f7e-8ee4-084c5095d9b2`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:52,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:52,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137926 s, 74.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:54,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:54,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0110742 s, 92.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:04:55,642::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3e0a2e8c] >Thread-18::DEBUG::2014-08-22 10:04:55,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`53447a76-aa47-4ec7-99d5-723d06314def`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:04:55,643::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:04:55,643::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:55,643::task::1191::Storage.TaskManager.Task::(prepare) Task=`53447a76-aa47-4ec7-99d5-723d06314def`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:04:55,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`53447a76-aa47-4ec7-99d5-723d06314def`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:04:55,643::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:04:55,643::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:04:55,643::task::993::Storage.TaskManager.Task::(_decref) Task=`53447a76-aa47-4ec7-99d5-723d06314def`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:04:55,681::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3e0a2e8c] >Thread-46::DEBUG::2014-08-22 10:04:55,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`4270b775-fc74-4513-8c7e-aac3a78ae2f4`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:04:55,681::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:04:55,681::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`474b7753-0abe-4890-8873-3a20756ecc0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:04:55,681::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:04:55,681::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:04:55,682::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`474b7753-0abe-4890-8873-3a20756ecc0e`::Granted request >Thread-46::DEBUG::2014-08-22 10:04:55,682::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4270b775-fc74-4513-8c7e-aac3a78ae2f4`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:04:55,682::task::993::Storage.TaskManager.Task::(_decref) Task=`4270b775-fc74-4513-8c7e-aac3a78ae2f4`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:04:55,684::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:55,684::task::1191::Storage.TaskManager.Task::(prepare) Task=`4270b775-fc74-4513-8c7e-aac3a78ae2f4`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:04:55,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`4270b775-fc74-4513-8c7e-aac3a78ae2f4`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:04:55,684::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:04:55,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:04:55,684::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:04:55,684::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:04:55,685::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:04:55,685::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:04:55,685::task::993::Storage.TaskManager.Task::(_decref) Task=`4270b775-fc74-4513-8c7e-aac3a78ae2f4`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:04:56,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:56,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112963 s, 90.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:04:58,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:04:58,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112486 s, 91.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:00,335::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:00,335::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:00,335::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:00,335::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:00,336::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 10:05:00,336::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:00,337::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:00,342::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:00,343::__init__::225::IOProcess::(_processLogs) (2179) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:00,343::__init__::225::IOProcess::(_processLogs) (2179) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:00,343::__init__::225::IOProcess::(_processLogs) (2179) Building response >Thread-17::DEBUG::2014-08-22 10:05:00,343::__init__::225::IOProcess::(_processLogs) (2179) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:00,343::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:00,343::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:05:00,344::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000211716 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:00,344::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:00,344::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:00,344::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) (2180) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) (2180) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) (2180) Building response >Thread-17::DEBUG::2014-08-22 10:05:00,345::__init__::225::IOProcess::(_processLogs) (2180) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:00,346::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:00,347::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:05:00,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:00,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115132 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:02,164::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:02,164::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:02,165::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:02,166::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:02,166::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:02,169::__init__::225::IOProcess::(_processLogs) (2181) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:02,170::__init__::225::IOProcess::(_processLogs) (2181) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 10:05:02,170::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:02,177::__init__::225::IOProcess::(_processLogs) (2181) Building response >Thread-17::DEBUG::2014-08-22 10:05:02,178::__init__::225::IOProcess::(_processLogs) (2181) Queuing response >Thread-22::DEBUG::2014-08-22 10:05:02,179::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000156643 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:02,180::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:02,181::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:02,182::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:02,183::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:02,183::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:02,184::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:02,185::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:02,185::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:02,186::__init__::225::IOProcess::(_processLogs) (2182) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:02,187::__init__::225::IOProcess::(_processLogs) (2182) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:02,190::__init__::225::IOProcess::(_processLogs) (2182) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:02,191::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:02,191::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:02,193::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:05:02,193::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:02,194::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:02,195::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:02,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:02,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:02,197::__init__::225::IOProcess::(_processLogs) (2183) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:02,197::__init__::225::IOProcess::(_processLogs) (2183) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:02,199::__init__::225::IOProcess::(_processLogs) (2183) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:02,200::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:02,200::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:02,202::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:05:02,202::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:02,203::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:02,204::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:02,204::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:02,205::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:02,205::__init__::225::IOProcess::(_processLogs) (2184) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:02,206::__init__::225::IOProcess::(_processLogs) (2184) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:02,208::__init__::225::IOProcess::(_processLogs) (2184) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:02,209::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:02,209::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:02,211::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:05:02,211::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:02,212::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:02,213::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:02,213::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:02,214::__init__::225::IOProcess::(_processLogs) (2185) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:02,215::__init__::225::IOProcess::(_processLogs) (2185) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:02,216::__init__::225::IOProcess::(_processLogs) (2185) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:02,217::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:02,218::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:05:02,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:02,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195111 s, 52.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:04,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:04,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114553 s, 89.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:05,734::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-18::DEBUG::2014-08-22 10:05:05,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`3800a46a-38c3-4405-a89a-de319e085ade`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:05,734::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:05:05,734::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:05,735::task::1191::Storage.TaskManager.Task::(prepare) Task=`3800a46a-38c3-4405-a89a-de319e085ade`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:05,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`3800a46a-38c3-4405-a89a-de319e085ade`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:05,735::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:05,735::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:05,735::task::993::Storage.TaskManager.Task::(_decref) Task=`3800a46a-38c3-4405-a89a-de319e085ade`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:05:05,773::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-46::DEBUG::2014-08-22 10:05:05,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc40cd01-e06d-498f-99d0-2eeb13610b4e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:05:05,773::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:05:05,773::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`800c4cf1-ef1b-47a8-8b93-2fabc9e99f47`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:05:05,774::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:05:05,774::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:05:05,774::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`800c4cf1-ef1b-47a8-8b93-2fabc9e99f47`::Granted request >Thread-46::DEBUG::2014-08-22 10:05:05,774::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fc40cd01-e06d-498f-99d0-2eeb13610b4e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:05:05,774::task::993::Storage.TaskManager.Task::(_decref) Task=`fc40cd01-e06d-498f-99d0-2eeb13610b4e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:05:05,776::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:05,776::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc40cd01-e06d-498f-99d0-2eeb13610b4e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:05,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc40cd01-e06d-498f-99d0-2eeb13610b4e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:05:05,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:05:05,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:05:05,777::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:05:05,777::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:05:05,777::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:05:05,777::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:05:05,777::task::993::Storage.TaskManager.Task::(_decref) Task=`fc40cd01-e06d-498f-99d0-2eeb13610b4e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:06,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:06,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162346 s, 63.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:07,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`450b233f-86c2-4f1f-916c-95c28ab5b598`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:07,395::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:05:07,395::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000211716', 'lastCheck': '7.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000156643', 'lastCheck': '5.2', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:07,395::task::1191::Storage.TaskManager.Task::(prepare) Task=`450b233f-86c2-4f1f-916c-95c28ab5b598`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000211716', 'lastCheck': '7.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000156643', 'lastCheck': '5.2', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:07,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`450b233f-86c2-4f1f-916c-95c28ab5b598`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:07,396::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:07,396::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:07,396::task::993::Storage.TaskManager.Task::(_decref) Task=`450b233f-86c2-4f1f-916c-95c28ab5b598`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:08,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:08,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011334 s, 90.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:10,347::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) (2186) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:10,348::__init__::225::IOProcess::(_processLogs) (2186) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:10,349::__init__::225::IOProcess::(_processLogs) (2186) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:10,349::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:05:10,349::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:10,350::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:05:10,356::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182616 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:10,357::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:10,357::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:10,357::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:10,357::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:10,357::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:10,357::__init__::225::IOProcess::(_processLogs) (2187) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:10,358::__init__::225::IOProcess::(_processLogs) (2187) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:10,358::__init__::225::IOProcess::(_processLogs) (2187) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:10,359::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:10,359::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:05:10,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:10,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157727 s, 64.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) (2188) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:12,218::__init__::225::IOProcess::(_processLogs) (2188) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:12,219::__init__::225::IOProcess::(_processLogs) (2188) Queuing response >Thread-22::DEBUG::2014-08-22 10:05:12,219::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:12,219::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:12,226::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:05:12,227::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000219787 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:12,227::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:12,227::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:12,227::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:12,228::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:12,228::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:12,228::__init__::225::IOProcess::(_processLogs) (2189) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:12,228::__init__::225::IOProcess::(_processLogs) (2189) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:12,228::__init__::225::IOProcess::(_processLogs) (2189) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:12,229::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:12,229::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:12,229::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:05:12,229::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:12,230::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:12,230::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:12,230::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:12,230::__init__::225::IOProcess::(_processLogs) (2190) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:12,230::__init__::225::IOProcess::(_processLogs) (2190) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:12,231::__init__::225::IOProcess::(_processLogs) (2190) Building response >Thread-17::DEBUG::2014-08-22 10:05:12,231::__init__::225::IOProcess::(_processLogs) (2190) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:12,231::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:12,231::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:12,231::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:05:12,231::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) (2191) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) (2191) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) (2191) Building response >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) (2191) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:12,232::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:12,233::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:05:12,233::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:12,233::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:12,233::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) (2192) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) (2192) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) (2192) Building response >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) (2192) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:12,234::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:05:12,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:12,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115541 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:14,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:15,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113326 s, 90.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:15,911::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:05:15,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8ff8683-c328-4395-a111-4fcbefae85a4`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:15,912::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:05:15,912::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:15,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8ff8683-c328-4395-a111-4fcbefae85a4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:15,913::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8ff8683-c328-4395-a111-4fcbefae85a4`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:15,913::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:15,913::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:15,913::task::993::Storage.TaskManager.Task::(_decref) Task=`a8ff8683-c328-4395-a111-4fcbefae85a4`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:05:15,954::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:05:15,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`b79da1c0-ffec-4f46-9fa3-13122d3b7583`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:05:15,954::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:05:15,955::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0a2ad2b2-b308-4936-8072-daa16daa6f2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:05:15,955::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:05:15,955::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:05:15,955::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0a2ad2b2-b308-4936-8072-daa16daa6f2e`::Granted request >Thread-46::DEBUG::2014-08-22 10:05:15,955::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b79da1c0-ffec-4f46-9fa3-13122d3b7583`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:05:15,955::task::993::Storage.TaskManager.Task::(_decref) Task=`b79da1c0-ffec-4f46-9fa3-13122d3b7583`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:05:15,957::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:15,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`b79da1c0-ffec-4f46-9fa3-13122d3b7583`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:15,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`b79da1c0-ffec-4f46-9fa3-13122d3b7583`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:05:15,958::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:05:15,958::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:05:15,958::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:05:15,958::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:05:15,958::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:05:15,958::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:05:15,958::task::993::Storage.TaskManager.Task::(_decref) Task=`b79da1c0-ffec-4f46-9fa3-13122d3b7583`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:17,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:17,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113719 s, 90.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:19,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:19,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112735 s, 90.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) (2193) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:20,360::__init__::225::IOProcess::(_processLogs) (2193) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:20,361::__init__::225::IOProcess::(_processLogs) (2193) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:20,361::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:05:20,362::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:20,362::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:05:20,369::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000195456 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:20,369::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:20,369::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:20,369::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:20,369::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:20,370::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:20,370::__init__::225::IOProcess::(_processLogs) (2194) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:20,370::__init__::225::IOProcess::(_processLogs) (2194) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:20,370::__init__::225::IOProcess::(_processLogs) (2194) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:20,371::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:20,371::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:05:21,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:21,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116602 s, 87.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:22,234::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:22,234::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:22,234::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:22,235::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:22,235::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:22,235::__init__::225::IOProcess::(_processLogs) (2195) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:22,235::__init__::225::IOProcess::(_processLogs) (2195) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:22,235::__init__::225::IOProcess::(_processLogs) (2195) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:22,235::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:05:22,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:05:22,242::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000225601 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:22,243::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:22,244::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:22,244::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:22,244::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:22,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:22,244::__init__::225::IOProcess::(_processLogs) (2196) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:22,244::__init__::225::IOProcess::(_processLogs) (2196) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:22,245::__init__::225::IOProcess::(_processLogs) (2196) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:22,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:22,245::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) (2197) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:22,246::__init__::225::IOProcess::(_processLogs) (2197) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:22,247::__init__::225::IOProcess::(_processLogs) (2197) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:22,247::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:22,247::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:22,247::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:05:22,247::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) (2198) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) (2198) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) (2198) Building response >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) (2198) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:22,248::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:22,249::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:05:22,249::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:22,249::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:22,249::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) (2199) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) (2199) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) (2199) Building response >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) (2199) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:22,250::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::DEBUG::2014-08-22 10:05:22,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0e11416-2f2b-43e8-bfeb-8f1afbdc42c3`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:22,693::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:05:22,693::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000195456', 'lastCheck': '2.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225601', 'lastCheck': '0.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:22,693::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0e11416-2f2b-43e8-bfeb-8f1afbdc42c3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000195456', 'lastCheck': '2.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225601', 'lastCheck': '0.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:22,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0e11416-2f2b-43e8-bfeb-8f1afbdc42c3`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:22,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:22,694::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:22,694::task::993::Storage.TaskManager.Task::(_decref) Task=`e0e11416-2f2b-43e8-bfeb-8f1afbdc42c3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:23,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:23,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119779 s, 85.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:25,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:25,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114761 s, 89.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:26,007::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:05:26,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`dadca31f-950b-4cae-a2cc-9ae5677ed561`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:26,007::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:05:26,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:26,008::task::1191::Storage.TaskManager.Task::(prepare) Task=`dadca31f-950b-4cae-a2cc-9ae5677ed561`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:26,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`dadca31f-950b-4cae-a2cc-9ae5677ed561`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:26,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:26,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:26,008::task::993::Storage.TaskManager.Task::(_decref) Task=`dadca31f-950b-4cae-a2cc-9ae5677ed561`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:05:26,032::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:05:26,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`f52d0646-4e8b-43e3-8177-95c6ea2cc2a0`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:05:26,032::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:05:26,033::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b68af398-0491-4233-a7d9-0edbd44434fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:05:26,033::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:05:26,033::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:05:26,033::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b68af398-0491-4233-a7d9-0edbd44434fa`::Granted request >Thread-46::DEBUG::2014-08-22 10:05:26,033::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f52d0646-4e8b-43e3-8177-95c6ea2cc2a0`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:05:26,033::task::993::Storage.TaskManager.Task::(_decref) Task=`f52d0646-4e8b-43e3-8177-95c6ea2cc2a0`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:05:26,035::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:26,035::task::1191::Storage.TaskManager.Task::(prepare) Task=`f52d0646-4e8b-43e3-8177-95c6ea2cc2a0`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:26,035::task::595::Storage.TaskManager.Task::(_updateState) Task=`f52d0646-4e8b-43e3-8177-95c6ea2cc2a0`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:05:26,035::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:05:26,036::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:05:26,036::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:05:26,036::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:05:26,036::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:05:26,036::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:05:26,036::task::993::Storage.TaskManager.Task::(_decref) Task=`f52d0646-4e8b-43e3-8177-95c6ea2cc2a0`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:27,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:27,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117174 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:29,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:29,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114457 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) (2200) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:30,372::__init__::225::IOProcess::(_processLogs) (2200) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:30,373::__init__::225::IOProcess::(_processLogs) (2200) Queuing response >Thread-21::DEBUG::2014-08-22 10:05:30,374::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:30,374::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:30,380::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:05:30,381::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182661 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) (2201) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:30,382::__init__::225::IOProcess::(_processLogs) (2201) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:30,383::__init__::225::IOProcess::(_processLogs) (2201) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:30,384::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:30,384::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:05:31,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:31,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116277 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:32,251::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:32,252::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:32,252::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:32,253::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:32,254::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:32,254::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:32,255::__init__::225::IOProcess::(_processLogs) (2202) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:32,256::__init__::225::IOProcess::(_processLogs) (2202) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:32,258::__init__::225::IOProcess::(_processLogs) (2202) Queuing response >Thread-22::DEBUG::2014-08-22 10:05:32,259::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:32,260::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:32,266::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:05:32,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000229718 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:32,270::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:32,270::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:32,271::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:32,272::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:32,272::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:32,273::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:32,273::__init__::225::IOProcess::(_processLogs) (2203) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:32,274::__init__::225::IOProcess::(_processLogs) (2203) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:32,277::__init__::225::IOProcess::(_processLogs) (2203) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:32,278::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:32,279::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:32,280::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:05:32,281::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:32,281::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:32,282::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:32,283::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:32,283::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:32,284::__init__::225::IOProcess::(_processLogs) (2204) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:32,285::__init__::225::IOProcess::(_processLogs) (2204) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:32,285::__init__::225::IOProcess::(_processLogs) (2204) Building response >Thread-17::DEBUG::2014-08-22 10:05:32,286::__init__::225::IOProcess::(_processLogs) (2204) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:32,288::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:32,289::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:05:32,290::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:32,290::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:32,291::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:32,292::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:32,293::__init__::225::IOProcess::(_processLogs) (2205) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:32,294::__init__::225::IOProcess::(_processLogs) (2205) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:32,294::__init__::225::IOProcess::(_processLogs) (2205) Building response >Thread-17::DEBUG::2014-08-22 10:05:32,295::__init__::225::IOProcess::(_processLogs) (2205) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:32,296::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:32,296::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:32,298::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:05:32,298::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:32,299::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:32,300::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:32,300::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:32,302::__init__::225::IOProcess::(_processLogs) (2206) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:32,303::__init__::225::IOProcess::(_processLogs) (2206) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:32,303::__init__::225::IOProcess::(_processLogs) (2206) Building response >Thread-17::DEBUG::2014-08-22 10:05:32,304::__init__::225::IOProcess::(_processLogs) (2206) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:32,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:32,305::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:05:33,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:33,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120553 s, 84.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:35,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:35,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112657 s, 90.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:36,118::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-18::DEBUG::2014-08-22 10:05:36,118::task::595::Storage.TaskManager.Task::(_updateState) Task=`c20354fd-6a4a-4f53-83c3-d2b0defb9b6d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:36,119::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:05:36,119::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:36,119::task::1191::Storage.TaskManager.Task::(prepare) Task=`c20354fd-6a4a-4f53-83c3-d2b0defb9b6d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:36,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`c20354fd-6a4a-4f53-83c3-d2b0defb9b6d`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:36,119::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:36,119::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:36,119::task::993::Storage.TaskManager.Task::(_decref) Task=`c20354fd-6a4a-4f53-83c3-d2b0defb9b6d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:05:36,152::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-46::DEBUG::2014-08-22 10:05:36,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d053721-af30-4450-b160-8cee9bb554a6`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:05:36,153::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:05:36,153::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1e569df4-14b8-48ba-ad40-197bb3e79342`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:05:36,153::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:05:36,153::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:05:36,153::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`1e569df4-14b8-48ba-ad40-197bb3e79342`::Granted request >Thread-46::DEBUG::2014-08-22 10:05:36,153::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7d053721-af30-4450-b160-8cee9bb554a6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:05:36,153::task::993::Storage.TaskManager.Task::(_decref) Task=`7d053721-af30-4450-b160-8cee9bb554a6`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:05:36,155::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:36,155::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d053721-af30-4450-b160-8cee9bb554a6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:36,156::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d053721-af30-4450-b160-8cee9bb554a6`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:05:36,156::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:05:36,156::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:05:36,156::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:05:36,156::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:05:36,156::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:05:36,156::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:05:36,156::task::993::Storage.TaskManager.Task::(_decref) Task=`7d053721-af30-4450-b160-8cee9bb554a6`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:37,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:37,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113117 s, 90.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:37,899::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd5da3e0-2976-4ee0-bd7d-4725f39054d7`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:37,900::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:05:37,900::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182661', 'lastCheck': '7.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000229718', 'lastCheck': '5.6', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:37,900::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd5da3e0-2976-4ee0-bd7d-4725f39054d7`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000182661', 'lastCheck': '7.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000229718', 'lastCheck': '5.6', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:37,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd5da3e0-2976-4ee0-bd7d-4725f39054d7`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:37,900::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:37,900::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:37,900::task::993::Storage.TaskManager.Task::(_decref) Task=`dd5da3e0-2976-4ee0-bd7d-4725f39054d7`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:39,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:39,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115402 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:40,384::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:40,384::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:40,385::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:40,385::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:40,385::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:40,385::__init__::225::IOProcess::(_processLogs) (2207) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:40,385::__init__::225::IOProcess::(_processLogs) (2207) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:40,386::__init__::225::IOProcess::(_processLogs) (2207) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:40,386::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:05:40,386::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:40,387::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:05:40,393::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000231984 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:40,394::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:40,394::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:40,394::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:40,394::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:40,394::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:40,395::__init__::225::IOProcess::(_processLogs) (2208) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:40,395::__init__::225::IOProcess::(_processLogs) (2208) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:40,395::__init__::225::IOProcess::(_processLogs) (2208) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:40,396::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:40,396::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:05:41,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:41,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114428 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:42,303::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:42,303::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:42,303::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:42,303::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:42,304::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:42,304::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:42,304::__init__::225::IOProcess::(_processLogs) (2209) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:42,304::__init__::225::IOProcess::(_processLogs) (2209) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:42,304::__init__::225::IOProcess::(_processLogs) DEBUG|(2209) Building response >Thread-17::DEBUG::2014-08-22 10:05:42,305::__init__::225::IOProcess::(_processLogs) (2209) Queuing response >Thread-22::DEBUG::2014-08-22 10:05:42,305::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:42,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:42,312::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:05:42,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218867 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:42,313::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:42,313::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) (2210) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:42,314::__init__::225::IOProcess::(_processLogs) (2210) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:42,315::__init__::225::IOProcess::(_processLogs) DEBUG|(2210) Building response >Thread-17::DEBUG::2014-08-22 10:05:42,315::__init__::225::IOProcess::(_processLogs) (2210) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:42,315::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:42,316::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:42,316::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:42,316::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:05:42,316::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:42,316::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) (2211) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) (2211) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) (2211) Building response >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) (2211) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:42,317::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:42,318::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:05:42,318::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:42,319::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:42,319::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:42,319::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:42,319::__init__::225::IOProcess::(_processLogs) (2212) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) (2212) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) (2212) Building response >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) (2212) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:05:42,320::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:42,321::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:42,321::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:42,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) (2213) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) (2213) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) (2213) Building response >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) (2213) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:42,322::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:05:43,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:43,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115636 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:45,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:45,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114449 s, 89.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:46,206::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:05:46,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3e12b88-cdc8-4181-8cff-23a9619d21a8`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:46,206::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:05:46,207::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:46,207::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3e12b88-cdc8-4181-8cff-23a9619d21a8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:46,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3e12b88-cdc8-4181-8cff-23a9619d21a8`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:46,207::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:46,207::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:46,207::task::993::Storage.TaskManager.Task::(_decref) Task=`d3e12b88-cdc8-4181-8cff-23a9619d21a8`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:05:46,247::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:05:46,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1609b1d-b6b2-4589-8082-b539facd3711`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:05:46,247::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:05:46,247::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`66630401-93fd-4703-9562-8675d4b6e2ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:05:46,247::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:05:46,247::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:05:46,247::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`66630401-93fd-4703-9562-8675d4b6e2ac`::Granted request >Thread-46::DEBUG::2014-08-22 10:05:46,248::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e1609b1d-b6b2-4589-8082-b539facd3711`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:05:46,248::task::993::Storage.TaskManager.Task::(_decref) Task=`e1609b1d-b6b2-4589-8082-b539facd3711`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:05:46,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:46,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1609b1d-b6b2-4589-8082-b539facd3711`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:46,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1609b1d-b6b2-4589-8082-b539facd3711`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:05:46,250::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:05:46,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:05:46,250::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:05:46,250::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:05:46,250::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:05:46,250::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:05:46,251::task::993::Storage.TaskManager.Task::(_decref) Task=`e1609b1d-b6b2-4589-8082-b539facd3711`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:47,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:47,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011461 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:49,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:49,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113618 s, 90.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) (2214) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:50,397::__init__::225::IOProcess::(_processLogs) (2214) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:50,398::__init__::225::IOProcess::(_processLogs) (2214) Queuing response >Thread-21::DEBUG::2014-08-22 10:05:50,399::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:50,399::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:50,406::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:05:50,406::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000209531 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) (2215) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:50,407::__init__::225::IOProcess::(_processLogs) (2215) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:50,408::__init__::225::IOProcess::(_processLogs) (2215) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:50,409::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:50,409::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:05:51,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:51,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115814 s, 88.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:52,322::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:52,322::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:52,322::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:52,322::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:52,322::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:52,322::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:52,323::__init__::225::IOProcess::(_processLogs) (2216) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:52,323::__init__::225::IOProcess::(_processLogs) (2216) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:52,323::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2216) Building response >Thread-22::DEBUG::2014-08-22 10:05:52,324::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:05:52,324::__init__::225::IOProcess::(_processLogs) (2216) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:52,330::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:52,331::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:05:52,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000190598 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:05:52,332::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) (2217) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) (2217) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) (2217) Building response >Thread-17::DEBUG::2014-08-22 10:05:52,333::__init__::225::IOProcess::(_processLogs) (2217) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:52,334::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:05:52,335::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:05:52,335::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:52,335::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:05:52,335::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:52,335::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) (2218) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) (2218) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) (2218) Building response >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) (2218) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:52,336::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:52,337::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:52,337::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:05:52,337::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:52,337::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:52,337::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) (2219) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) (2219) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) (2219) Building response >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) (2219) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:05:52,338::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:05:52,339::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:05:52,339::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:05:52,339::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:05:52,339::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) (2220) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) (2220) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) (2220) Building response >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) (2220) Queuing response >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:05:52,340::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::DEBUG::2014-08-22 10:05:53,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`f238c1b7-0417-40de-a253-315ddda49a64`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:53,138::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:05:53,138::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000209531', 'lastCheck': '2.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000190598', 'lastCheck': '0.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:53,138::task::1191::Storage.TaskManager.Task::(prepare) Task=`f238c1b7-0417-40de-a253-315ddda49a64`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000209531', 'lastCheck': '2.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000190598', 'lastCheck': '0.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:05:53,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`f238c1b7-0417-40de-a253-315ddda49a64`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:53,139::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:53,139::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:53,139::task::993::Storage.TaskManager.Task::(_decref) Task=`f238c1b7-0417-40de-a253-315ddda49a64`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:53,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:53,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114517 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:05:55,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:56,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112352 s, 91.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:05:56,310::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:05:56,310::task::595::Storage.TaskManager.Task::(_updateState) Task=`f91f6180-ded3-446c-8f51-bf73e65b06fd`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:05:56,310::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:05:56,311::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:56,311::task::1191::Storage.TaskManager.Task::(prepare) Task=`f91f6180-ded3-446c-8f51-bf73e65b06fd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:05:56,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`f91f6180-ded3-446c-8f51-bf73e65b06fd`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:05:56,311::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:05:56,311::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:05:56,311::task::993::Storage.TaskManager.Task::(_decref) Task=`f91f6180-ded3-446c-8f51-bf73e65b06fd`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:05:56,394::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:05:56,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`18c3551c-b7cb-4489-9dee-87a1f5c5d578`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:05:56,394::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:05:56,394::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c051f948-2e9a-44bc-9d43-553868b54b38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:05:56,394::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:05:56,395::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:05:56,395::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c051f948-2e9a-44bc-9d43-553868b54b38`::Granted request >Thread-46::DEBUG::2014-08-22 10:05:56,395::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`18c3551c-b7cb-4489-9dee-87a1f5c5d578`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:05:56,395::task::993::Storage.TaskManager.Task::(_decref) Task=`18c3551c-b7cb-4489-9dee-87a1f5c5d578`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:05:56,397::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:56,397::task::1191::Storage.TaskManager.Task::(prepare) Task=`18c3551c-b7cb-4489-9dee-87a1f5c5d578`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:05:56,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`18c3551c-b7cb-4489-9dee-87a1f5c5d578`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:05:56,397::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:05:56,397::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:05:56,397::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:05:56,398::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:05:56,398::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:05:56,398::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:05:56,398::task::993::Storage.TaskManager.Task::(_decref) Task=`18c3551c-b7cb-4489-9dee-87a1f5c5d578`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:05:58,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:05:58,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111807 s, 91.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:00,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:00,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115403 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) (2221) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:00,410::__init__::225::IOProcess::(_processLogs) (2221) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:00,411::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2221) Building response >Thread-17::DEBUG::2014-08-22 10:06:00,412::__init__::225::IOProcess::(_processLogs) (2221) Queuing response >Thread-21::DEBUG::2014-08-22 10:06:00,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:00,412::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:00,419::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:06:00,419::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000206148 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:00,420::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:00,420::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) (2222) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:00,421::__init__::225::IOProcess::(_processLogs) (2222) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:00,422::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2222) Building response >Thread-17::DEBUG::2014-08-22 10:06:00,423::__init__::225::IOProcess::(_processLogs) (2222) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:00,423::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:00,423::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:06:02,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:02,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219949 s, 46.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:02,341::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:02,342::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:02,342::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:02,343::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:02,343::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:02,344::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:02,347::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-22::DEBUG::2014-08-22 10:06:02,348::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:02,348::__init__::225::IOProcess::(_processLogs) (2223) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:02,355::__init__::225::IOProcess::(_processLogs) (2223) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 10:06:02,356::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000219073 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:02,356::__init__::225::IOProcess::(_processLogs) (2223) Building response >Thread-17::DEBUG::2014-08-22 10:06:02,358::__init__::225::IOProcess::(_processLogs) (2223) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:02,358::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:02,358::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:02,360::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:02,360::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:02,361::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:02,363::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:02,363::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:02,364::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:02,364::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:02,365::__init__::225::IOProcess::(_processLogs) (2224) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:02,365::__init__::225::IOProcess::(_processLogs) (2224) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:02,366::__init__::225::IOProcess::(_processLogs) (2224) Building response >Thread-17::DEBUG::2014-08-22 10:06:02,366::__init__::225::IOProcess::(_processLogs) (2224) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:02,367::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:02,367::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:02,368::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:02,369::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:06:02,369::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:02,370::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:02,370::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:02,371::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:02,372::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:02,372::__init__::225::IOProcess::(_processLogs) (2225) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:02,373::__init__::225::IOProcess::(_processLogs) (2225) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:02,373::__init__::225::IOProcess::(_processLogs) (2225) Building response >Thread-17::DEBUG::2014-08-22 10:06:02,373::__init__::225::IOProcess::(_processLogs) (2225) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:02,374::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:02,374::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:02,375::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:02,376::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:06:02,376::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:02,377::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:02,377::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:02,378::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:02,379::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:02,379::__init__::225::IOProcess::(_processLogs) (2226) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:02,380::__init__::225::IOProcess::(_processLogs) (2226) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:02,380::__init__::225::IOProcess::(_processLogs) (2226) Building response >Thread-17::DEBUG::2014-08-22 10:06:02,381::__init__::225::IOProcess::(_processLogs) (2226) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:02,381::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:02,381::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:02,382::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:02,383::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:06:02,383::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:02,384::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:02,385::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:02,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:02,386::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:02,387::__init__::225::IOProcess::(_processLogs) (2227) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:02,387::__init__::225::IOProcess::(_processLogs) (2227) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:02,388::__init__::225::IOProcess::(_processLogs) (2227) Building response >Thread-17::DEBUG::2014-08-22 10:06:02,388::__init__::225::IOProcess::(_processLogs) (2227) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:02,389::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:02,389::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:06:04,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:04,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123818 s, 82.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:06,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:06,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114749 s, 89.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:06,480::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:06:06,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`70f4bf7f-38fc-4550-9188-e0807f73cc45`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:06,480::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:06:06,481::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:06,481::task::1191::Storage.TaskManager.Task::(prepare) Task=`70f4bf7f-38fc-4550-9188-e0807f73cc45`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:06,481::task::595::Storage.TaskManager.Task::(_updateState) Task=`70f4bf7f-38fc-4550-9188-e0807f73cc45`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:06,481::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:06,481::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:06,481::task::993::Storage.TaskManager.Task::(_decref) Task=`70f4bf7f-38fc-4550-9188-e0807f73cc45`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:06:06,515::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:06:06,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`add2fbdf-f0dd-4b2d-a4f9-07a481c4b33d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:06:06,515::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:06:06,516::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`510fcf4f-9a54-4e1a-97d6-ff7140b9b7f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:06:06,516::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:06:06,516::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:06:06,516::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`510fcf4f-9a54-4e1a-97d6-ff7140b9b7f3`::Granted request >Thread-46::DEBUG::2014-08-22 10:06:06,516::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`add2fbdf-f0dd-4b2d-a4f9-07a481c4b33d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:06:06,516::task::993::Storage.TaskManager.Task::(_decref) Task=`add2fbdf-f0dd-4b2d-a4f9-07a481c4b33d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:06:06,518::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:06,518::task::1191::Storage.TaskManager.Task::(prepare) Task=`add2fbdf-f0dd-4b2d-a4f9-07a481c4b33d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:06,518::task::595::Storage.TaskManager.Task::(_updateState) Task=`add2fbdf-f0dd-4b2d-a4f9-07a481c4b33d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:06:06,519::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:06:06,519::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:06:06,519::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:06:06,519::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:06:06,519::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:06:06,519::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:06:06,519::task::993::Storage.TaskManager.Task::(_decref) Task=`add2fbdf-f0dd-4b2d-a4f9-07a481c4b33d`::ref 0 aborting False >Thread-18::DEBUG::2014-08-22 10:06:08,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`064fa246-ab65-4788-b642-ef9c3c3741d1`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:08,325::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:06:08,325::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206148', 'lastCheck': '7.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219073', 'lastCheck': '5.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:08,325::task::1191::Storage.TaskManager.Task::(prepare) Task=`064fa246-ab65-4788-b642-ef9c3c3741d1`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000206148', 'lastCheck': '7.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219073', 'lastCheck': '5.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:08,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`064fa246-ab65-4788-b642-ef9c3c3741d1`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:08,325::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:08,325::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:08,325::task::993::Storage.TaskManager.Task::(_decref) Task=`064fa246-ab65-4788-b642-ef9c3c3741d1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:08,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:08,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115373 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:10,423::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:10,423::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:10,423::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) (2228) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) (2228) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) (2228) Building response >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) (2228) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:10,424::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:06:10,425::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 10:06:10,431::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000229762 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) (2229) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:10,432::__init__::225::IOProcess::(_processLogs) (2229) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:10,433::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2229) Building response >Thread-17::DEBUG::2014-08-22 10:06:10,434::__init__::225::IOProcess::(_processLogs) (2229) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:10,434::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:10,434::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:06:10,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:10,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115475 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:12,386::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:12,386::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:12,386::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:12,386::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:12,387::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:12,387::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:12,387::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:12,387::__init__::225::IOProcess::(_processLogs) (2230) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:12,387::__init__::225::IOProcess::(_processLogs) (2230) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:12,387::__init__::225::IOProcess::(_processLogs) DEBUG|(2230) Building response >Thread-17::DEBUG::2014-08-22 10:06:12,388::__init__::225::IOProcess::(_processLogs) (2230) Queuing response >Thread-22::DEBUG::2014-08-22 10:06:12,388::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:12,389::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:12,395::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:06:12,395::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000182952 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:12,396::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:12,396::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:12,396::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:12,396::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:12,396::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:12,396::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:12,397::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:12,397::__init__::225::IOProcess::(_processLogs) (2231) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:12,397::__init__::225::IOProcess::(_processLogs) (2231) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:12,397::__init__::225::IOProcess::(_processLogs) DEBUG|(2231) Building response >Thread-17::DEBUG::2014-08-22 10:06:12,398::__init__::225::IOProcess::(_processLogs) (2231) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:12,398::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:12,398::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:12,398::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:12,398::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:06:12,399::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:12,399::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:12,399::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:12,399::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) (2232) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) (2232) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) (2232) Building response >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) (2232) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:06:12,400::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) (2233) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) (2233) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:12,401::__init__::225::IOProcess::(_processLogs) (2233) Building response >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) (2233) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:12,402::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:12,403::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:12,403::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:12,403::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:12,403::__init__::225::IOProcess::(_processLogs) (2234) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:12,403::__init__::225::IOProcess::(_processLogs) (2234) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:12,403::__init__::225::IOProcess::(_processLogs) (2234) Building response >Thread-17::DEBUG::2014-08-22 10:06:12,404::__init__::225::IOProcess::(_processLogs) (2234) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:12,404::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:12,404::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:06:12,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:12,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113821 s, 90.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:14,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:14,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119102 s, 86.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:16,585::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:06:16,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`de5b8c5e-db83-4926-bae5-ceec5f48aaaf`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:16,586::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:06:16,586::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:16,586::task::1191::Storage.TaskManager.Task::(prepare) Task=`de5b8c5e-db83-4926-bae5-ceec5f48aaaf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:16,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`de5b8c5e-db83-4926-bae5-ceec5f48aaaf`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:16,586::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:16,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:16,587::task::993::Storage.TaskManager.Task::(_decref) Task=`de5b8c5e-db83-4926-bae5-ceec5f48aaaf`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:16,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:16,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111128 s, 92.1 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 10:06:16,712::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:06:16,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e9685b6-a3fb-4ad6-adee-73e11a688ffc`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:06:16,712::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:06:16,713::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`26c8a83d-84a3-4a3b-b459-f5d9b2a73778`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:06:16,713::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:06:16,713::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:06:16,713::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`26c8a83d-84a3-4a3b-b459-f5d9b2a73778`::Granted request >Thread-46::DEBUG::2014-08-22 10:06:16,713::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0e9685b6-a3fb-4ad6-adee-73e11a688ffc`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:06:16,713::task::993::Storage.TaskManager.Task::(_decref) Task=`0e9685b6-a3fb-4ad6-adee-73e11a688ffc`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:06:16,715::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:16,715::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e9685b6-a3fb-4ad6-adee-73e11a688ffc`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:16,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e9685b6-a3fb-4ad6-adee-73e11a688ffc`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:06:16,716::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:06:16,716::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:06:16,716::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:06:16,716::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:06:16,716::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:06:16,716::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:06:16,716::task::993::Storage.TaskManager.Task::(_decref) Task=`0e9685b6-a3fb-4ad6-adee-73e11a688ffc`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:18,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:18,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117597 s, 87.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:20,434::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:20,434::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:20,434::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:20,435::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:20,435::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:20,435::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:20,435::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:20,435::__init__::225::IOProcess::(_processLogs) (2235) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:20,435::__init__::225::IOProcess::(_processLogs) (2235) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:20,436::__init__::225::IOProcess::(_processLogs) DEBUG|(2235) Building response >Thread-21::DEBUG::2014-08-22 10:06:20,436::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:20,436::__init__::225::IOProcess::(_processLogs) (2235) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:20,443::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:06:20,444::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000191285 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:20,444::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:20,445::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:20,445::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:20,445::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:20,446::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:20,446::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:20,446::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:20,446::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:20,446::__init__::225::IOProcess::(_processLogs) (2236) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:20,446::__init__::225::IOProcess::(_processLogs) (2236) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:20,447::__init__::225::IOProcess::(_processLogs) DEBUG|(2236) Building response >Thread-17::DEBUG::2014-08-22 10:06:20,447::__init__::225::IOProcess::(_processLogs) (2236) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:20,448::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:20,448::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:06:20,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:20,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116441 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) (2237) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:22,404::__init__::225::IOProcess::(_processLogs) (2237) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:22,405::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2237) Building response >Thread-17::DEBUG::2014-08-22 10:06:22,406::__init__::225::IOProcess::(_processLogs) (2237) Queuing response >Thread-22::DEBUG::2014-08-22 10:06:22,406::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:22,406::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:22,413::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:06:22,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198338 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:22,414::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:22,414::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:22,414::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:22,414::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:22,415::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:22,415::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:22,415::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:22,415::__init__::225::IOProcess::(_processLogs) (2238) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:22,415::__init__::225::IOProcess::(_processLogs) (2238) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:22,416::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2238) Building response >Thread-17::DEBUG::2014-08-22 10:06:22,416::__init__::225::IOProcess::(_processLogs) (2238) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:22,416::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:22,416::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:22,417::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:22,417::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:06:22,417::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:22,417::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:22,417::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) (2239) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) (2239) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) (2239) Building response >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) (2239) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:22,418::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:22,419::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:22,419::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:22,419::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:06:22,419::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) (2240) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) (2240) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:22,420::__init__::225::IOProcess::(_processLogs) (2240) Building response >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) (2240) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:22,421::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:22,422::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:22,422::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:22,422::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:22,422::__init__::225::IOProcess::(_processLogs) (2241) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:22,422::__init__::225::IOProcess::(_processLogs) (2241) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:22,423::__init__::225::IOProcess::(_processLogs) (2241) Building response >Thread-17::DEBUG::2014-08-22 10:06:22,423::__init__::225::IOProcess::(_processLogs) (2241) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:22,423::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:22,423::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:06:22,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:22,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121389 s, 84.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:23,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`be8afe70-3700-4477-9182-8210b456ed12`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:23,516::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:06:23,516::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000191285', 'lastCheck': '3.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198338', 'lastCheck': '1.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:23,516::task::1191::Storage.TaskManager.Task::(prepare) Task=`be8afe70-3700-4477-9182-8210b456ed12`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000191285', 'lastCheck': '3.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198338', 'lastCheck': '1.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:23,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`be8afe70-3700-4477-9182-8210b456ed12`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:23,517::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:23,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:23,517::task::993::Storage.TaskManager.Task::(_decref) Task=`be8afe70-3700-4477-9182-8210b456ed12`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:24,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:24,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011533 s, 88.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:26,792::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:06:26,792::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b375142-ae12-4b59-a358-a8151ca845d1`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:26,793::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:06:26,793::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:26,793::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b375142-ae12-4b59-a358-a8151ca845d1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:26,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b375142-ae12-4b59-a358-a8151ca845d1`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:26,793::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:26,793::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:26,793::task::993::Storage.TaskManager.Task::(_decref) Task=`6b375142-ae12-4b59-a358-a8151ca845d1`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:06:26,831::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:06:26,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0cf6f43-8cd6-4af1-ada1-9628c93b14df`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:06:26,831::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:06:26,831::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9081cc1b-e57c-4157-bc56-cbac4de89071`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:06:26,831::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:06:26,832::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:06:26,832::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9081cc1b-e57c-4157-bc56-cbac4de89071`::Granted request >Thread-46::DEBUG::2014-08-22 10:06:26,832::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f0cf6f43-8cd6-4af1-ada1-9628c93b14df`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:06:26,832::task::993::Storage.TaskManager.Task::(_decref) Task=`f0cf6f43-8cd6-4af1-ada1-9628c93b14df`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:06:26,834::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:26,834::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0cf6f43-8cd6-4af1-ada1-9628c93b14df`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:26,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0cf6f43-8cd6-4af1-ada1-9628c93b14df`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:06:26,834::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:06:26,834::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:06:26,834::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:06:26,835::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:06:26,835::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:06:26,835::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:06:26,835::task::993::Storage.TaskManager.Task::(_decref) Task=`f0cf6f43-8cd6-4af1-ada1-9628c93b14df`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:26,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:26,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113587 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:28,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:28,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114338 s, 89.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:30,448::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:30,448::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:30,448::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:30,449::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:06:30,449::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:30,450::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:30,456::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:06:30,457::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227739 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:30,457::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:30,458::__init__::225::IOProcess::(_processLogs) (2242) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:30,458::__init__::225::IOProcess::(_processLogs) (2242) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:30,458::__init__::225::IOProcess::(_processLogs) (2242) Building response >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) (2242) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:30,459::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:30,460::__init__::225::IOProcess::(_processLogs) (2243) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:30,460::__init__::225::IOProcess::(_processLogs) (2243) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:30,460::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2243) Building response >Thread-17::DEBUG::2014-08-22 10:06:30,461::__init__::225::IOProcess::(_processLogs) (2243) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:30,461::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:30,461::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:06:30,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:30,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116341 s, 88.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:32,423::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:32,424::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:32,425::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:32,425::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:32,428::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:32,429::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:06:32,429::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:32,436::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:32,437::__init__::225::IOProcess::(_processLogs) (2244) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:32,438::__init__::225::IOProcess::(_processLogs) (2244) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 10:06:32,438::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000233194 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:32,439::__init__::225::IOProcess::(_processLogs) (2244) Building response >Thread-17::DEBUG::2014-08-22 10:06:32,440::__init__::225::IOProcess::(_processLogs) (2244) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:32,441::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:32,441::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:32,442::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:32,443::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:32,443::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:32,445::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:32,446::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:32,447::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:32,447::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:32,448::__init__::225::IOProcess::(_processLogs) (2245) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:32,448::__init__::225::IOProcess::(_processLogs) (2245) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:32,449::__init__::225::IOProcess::(_processLogs) (2245) Building response >Thread-17::DEBUG::2014-08-22 10:06:32,449::__init__::225::IOProcess::(_processLogs) (2245) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:32,450::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:32,450::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:32,451::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:32,452::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:06:32,452::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:32,453::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:32,454::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:32,454::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:32,455::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:32,455::__init__::225::IOProcess::(_processLogs) (2246) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:32,456::__init__::225::IOProcess::(_processLogs) (2246) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:32,456::__init__::225::IOProcess::(_processLogs) (2246) Building response >Thread-17::DEBUG::2014-08-22 10:06:32,457::__init__::225::IOProcess::(_processLogs) (2246) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:32,457::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:32,458::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:32,458::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:32,459::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:06:32,459::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:32,460::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:32,460::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:32,461::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:32,462::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:32,463::__init__::225::IOProcess::(_processLogs) (2247) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:32,463::__init__::225::IOProcess::(_processLogs) (2247) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:32,463::__init__::225::IOProcess::(_processLogs) (2247) Building response >Thread-17::DEBUG::2014-08-22 10:06:32,464::__init__::225::IOProcess::(_processLogs) (2247) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:32,464::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:32,465::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:32,466::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:32,466::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:06:32,467::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:32,467::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:32,468::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:32,469::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:32,470::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:32,470::__init__::225::IOProcess::(_processLogs) (2248) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:32,471::__init__::225::IOProcess::(_processLogs) (2248) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:32,471::__init__::225::IOProcess::(_processLogs) (2248) Building response >Thread-17::DEBUG::2014-08-22 10:06:32,472::__init__::225::IOProcess::(_processLogs) (2248) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:32,472::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:32,473::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:06:32,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:33,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118492 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:35,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:35,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115753 s, 88.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:36,883::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:06:36,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5c66821-a1b7-4c84-bccb-b0f08ea2d2e4`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:36,883::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:06:36,884::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:36,884::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5c66821-a1b7-4c84-bccb-b0f08ea2d2e4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:36,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5c66821-a1b7-4c84-bccb-b0f08ea2d2e4`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:36,884::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:36,884::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:36,884::task::993::Storage.TaskManager.Task::(_decref) Task=`d5c66821-a1b7-4c84-bccb-b0f08ea2d2e4`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:06:36,917::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:06:36,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`741e15d1-ca59-4343-a286-bfcd679bf58b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:06:36,917::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:06:36,917::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3530ffbe-fc40-4e3f-9d72-7a43075a678e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:06:36,917::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:06:36,917::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:06:36,917::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3530ffbe-fc40-4e3f-9d72-7a43075a678e`::Granted request >Thread-46::DEBUG::2014-08-22 10:06:36,918::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`741e15d1-ca59-4343-a286-bfcd679bf58b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:06:36,918::task::993::Storage.TaskManager.Task::(_decref) Task=`741e15d1-ca59-4343-a286-bfcd679bf58b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:06:36,920::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:36,920::task::1191::Storage.TaskManager.Task::(prepare) Task=`741e15d1-ca59-4343-a286-bfcd679bf58b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:36,920::task::595::Storage.TaskManager.Task::(_updateState) Task=`741e15d1-ca59-4343-a286-bfcd679bf58b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:06:36,920::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:06:36,920::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:06:36,920::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:06:36,920::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:06:36,920::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:06:36,920::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:06:36,921::task::993::Storage.TaskManager.Task::(_decref) Task=`741e15d1-ca59-4343-a286-bfcd679bf58b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:37,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:37,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111641 s, 91.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:38,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9bb5320-fbf3-475c-8048-30bcd8e44691`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:38,758::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:06:38,758::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227739', 'lastCheck': '8.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000233194', 'lastCheck': '6.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:38,758::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9bb5320-fbf3-475c-8048-30bcd8e44691`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000227739', 'lastCheck': '8.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000233194', 'lastCheck': '6.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:38,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9bb5320-fbf3-475c-8048-30bcd8e44691`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:38,759::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:38,759::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:38,759::task::993::Storage.TaskManager.Task::(_decref) Task=`c9bb5320-fbf3-475c-8048-30bcd8e44691`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:39,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:39,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117212 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:40,462::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:40,462::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:40,462::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:40,462::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:06:40,463::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:40,463::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:40,470::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:06:40,471::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000227891 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:40,471::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:40,472::__init__::225::IOProcess::(_processLogs) (2249) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:40,472::__init__::225::IOProcess::(_processLogs) (2249) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:40,472::__init__::225::IOProcess::(_processLogs) (2249) Building response >Thread-17::DEBUG::2014-08-22 10:06:40,472::__init__::225::IOProcess::(_processLogs) (2249) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:40,472::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:40,472::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) (2250) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:40,473::__init__::225::IOProcess::(_processLogs) (2250) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:40,474::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2250) Building response >Thread-17::DEBUG::2014-08-22 10:06:40,475::__init__::225::IOProcess::(_processLogs) (2250) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:40,475::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:40,475::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:06:41,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:41,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116217 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) (2251) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:42,470::__init__::225::IOProcess::(_processLogs) (2251) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:42,471::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2251) Building response >Thread-22::DEBUG::2014-08-22 10:06:42,472::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:42,472::__init__::225::IOProcess::(_processLogs) (2251) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:42,478::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:42,479::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:06:42,479::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000207582 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:42,480::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:42,480::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) (2252) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) (2252) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) (2252) Building response >Thread-17::DEBUG::2014-08-22 10:06:42,481::__init__::225::IOProcess::(_processLogs) (2252) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:42,482::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:06:42,483::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:42,483::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:42,483::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:06:42,483::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:42,483::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) (2253) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) (2253) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) (2253) Building response >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) (2253) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:42,484::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:42,485::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:42,485::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:06:42,485::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:42,485::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:42,485::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:42,485::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) (2254) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) (2254) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) (2254) Building response >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) (2254) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:06:42,486::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:42,487::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:42,487::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:42,487::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:42,487::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:42,488::__init__::225::IOProcess::(_processLogs) (2255) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:42,488::__init__::225::IOProcess::(_processLogs) (2255) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:42,488::__init__::225::IOProcess::(_processLogs) (2255) Building response >Thread-17::DEBUG::2014-08-22 10:06:42,488::__init__::225::IOProcess::(_processLogs) (2255) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:42,488::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:42,488::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:06:43,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:43,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115056 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:45,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:45,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114592 s, 89.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:46,987::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-18::DEBUG::2014-08-22 10:06:46,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`75543563-3ef8-4182-975d-f84ceea8c8ba`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:46,987::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:06:46,987::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:46,987::task::1191::Storage.TaskManager.Task::(prepare) Task=`75543563-3ef8-4182-975d-f84ceea8c8ba`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:46,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`75543563-3ef8-4182-975d-f84ceea8c8ba`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:46,988::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:46,988::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:46,988::task::993::Storage.TaskManager.Task::(_decref) Task=`75543563-3ef8-4182-975d-f84ceea8c8ba`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:06:47,021::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [11d355bd] >Thread-46::DEBUG::2014-08-22 10:06:47,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c3340e3-e3b9-42c1-9d33-bcf6460431d1`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:06:47,021::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:06:47,021::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5ae60086-c678-451b-9465-3598d36a96e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:06:47,021::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:06:47,021::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:06:47,021::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`5ae60086-c678-451b-9465-3598d36a96e3`::Granted request >Thread-46::DEBUG::2014-08-22 10:06:47,022::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8c3340e3-e3b9-42c1-9d33-bcf6460431d1`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:06:47,022::task::993::Storage.TaskManager.Task::(_decref) Task=`8c3340e3-e3b9-42c1-9d33-bcf6460431d1`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:06:47,024::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:47,024::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c3340e3-e3b9-42c1-9d33-bcf6460431d1`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:47,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c3340e3-e3b9-42c1-9d33-bcf6460431d1`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:06:47,024::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:06:47,024::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:06:47,024::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:06:47,024::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:06:47,024::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:06:47,024::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:06:47,025::task::993::Storage.TaskManager.Task::(_decref) Task=`8c3340e3-e3b9-42c1-9d33-bcf6460431d1`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:47,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:47,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112816 s, 90.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:49,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:49,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116238 s, 88.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:50,475::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) (2256) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:50,476::__init__::225::IOProcess::(_processLogs) (2256) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:50,477::__init__::225::IOProcess::(_processLogs) DEBUG|(2256) Building response >Thread-17::DEBUG::2014-08-22 10:06:50,478::__init__::225::IOProcess::(_processLogs) (2256) Queuing response >Thread-21::DEBUG::2014-08-22 10:06:50,477::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:50,478::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:50,485::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:06:50,485::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000171441 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:50,486::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:50,486::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:06:50,486::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:50,486::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:50,486::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:50,486::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:50,487::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:50,487::__init__::225::IOProcess::(_processLogs) (2257) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:50,487::__init__::225::IOProcess::(_processLogs) (2257) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:50,487::__init__::225::IOProcess::(_processLogs) DEBUG|(2257) Building response >Thread-17::DEBUG::2014-08-22 10:06:50,488::__init__::225::IOProcess::(_processLogs) (2257) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:50,488::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:50,488::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:06:51,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:51,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120516 s, 85.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:52,488::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:52,489::__init__::225::IOProcess::(_processLogs) (2258) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:52,489::__init__::225::IOProcess::(_processLogs) (2258) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:52,489::__init__::225::IOProcess::(_processLogs) DEBUG|(2258) Building response >Thread-22::DEBUG::2014-08-22 10:06:52,490::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:06:52,490::__init__::225::IOProcess::(_processLogs) (2258) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:52,496::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:06:52,497::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000208854 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:06:52,498::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:52,499::__init__::225::IOProcess::(_processLogs) (2259) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:06:52,500::__init__::225::IOProcess::(_processLogs) (2259) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:06:52,500::__init__::225::IOProcess::(_processLogs) DEBUG|(2259) Building response >Thread-17::DEBUG::2014-08-22 10:06:52,501::__init__::225::IOProcess::(_processLogs) (2259) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:52,501::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:52,501::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:06:52,501::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:52,501::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:06:52,501::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) (2260) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) (2260) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) (2260) Building response >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) (2260) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:52,502::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:52,503::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:52,503::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:52,503::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:06:52,503::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:52,503::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) (2261) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) (2261) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) (2261) Building response >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) (2261) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:06:52,504::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:06:52,505::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:06:52,505::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:06:52,505::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) (2262) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) (2262) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) (2262) Building response >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) (2262) Queuing response >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:06:52,506::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:06:53,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:53,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116055 s, 88.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:53,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`093d1438-cb71-4d6b-ac61-fef17dbdfd3a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:53,975::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:06:53,975::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000171441', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000208854', 'lastCheck': '1.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:53,975::task::1191::Storage.TaskManager.Task::(prepare) Task=`093d1438-cb71-4d6b-ac61-fef17dbdfd3a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000171441', 'lastCheck': '3.5', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000208854', 'lastCheck': '1.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:06:53,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`093d1438-cb71-4d6b-ac61-fef17dbdfd3a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:53,975::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:53,975::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:53,976::task::993::Storage.TaskManager.Task::(_decref) Task=`093d1438-cb71-4d6b-ac61-fef17dbdfd3a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:55,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:55,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115935 s, 88.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:06:57,073::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:06:57,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`10265081-d50a-4237-b062-ffc6624db096`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:06:57,074::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:06:57,074::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:57,074::task::1191::Storage.TaskManager.Task::(prepare) Task=`10265081-d50a-4237-b062-ffc6624db096`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:06:57,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`10265081-d50a-4237-b062-ffc6624db096`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:06:57,074::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:06:57,075::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:06:57,075::task::993::Storage.TaskManager.Task::(_decref) Task=`10265081-d50a-4237-b062-ffc6624db096`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:06:57,108::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:06:57,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6525667-0091-4c1a-bb9d-08e0e3d1ae22`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:06:57,108::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:06:57,108::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a95967ee-ca24-4acc-9acf-267ad472d891`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:06:57,108::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:06:57,109::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:06:57,109::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a95967ee-ca24-4acc-9acf-267ad472d891`::Granted request >Thread-46::DEBUG::2014-08-22 10:06:57,109::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6525667-0091-4c1a-bb9d-08e0e3d1ae22`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:06:57,109::task::993::Storage.TaskManager.Task::(_decref) Task=`b6525667-0091-4c1a-bb9d-08e0e3d1ae22`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:06:57,111::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:57,111::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6525667-0091-4c1a-bb9d-08e0e3d1ae22`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:06:57,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6525667-0091-4c1a-bb9d-08e0e3d1ae22`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:06:57,111::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:06:57,111::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:06:57,111::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:06:57,112::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:06:57,112::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:06:57,112::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:06:57,112::task::993::Storage.TaskManager.Task::(_decref) Task=`b6525667-0091-4c1a-bb9d-08e0e3d1ae22`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:06:57,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:57,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137828 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:06:59,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:06:59,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113441 s, 90.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) (2263) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) (2263) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:00,489::__init__::225::IOProcess::(_processLogs) (2263) Building response >Thread-17::DEBUG::2014-08-22 10:07:00,490::__init__::225::IOProcess::(_processLogs) (2263) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:00,490::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:00,490::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:07:00,491::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 10:07:00,498::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000283875 s, 1.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:00,498::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:00,498::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:00,498::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:00,499::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) (2264) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) (2264) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) (2264) Building response >Thread-17::DEBUG::2014-08-22 10:07:00,500::__init__::225::IOProcess::(_processLogs) (2264) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:00,501::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:00,501::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:07:01,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:01,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114981 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:02,507::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:02,508::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:02,508::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:02,509::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:02,510::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:02,510::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:02,511::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:02,512::__init__::225::IOProcess::(_processLogs) (2265) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:02,513::__init__::225::IOProcess::(_processLogs) (2265) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:02,516::__init__::225::IOProcess::(_processLogs) DEBUG|(2265) Building response >Thread-22::DEBUG::2014-08-22 10:07:02,517::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:02,517::__init__::225::IOProcess::(_processLogs) (2265) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:02,524::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:02,525::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:07:02,525::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000209704 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:02,528::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:02,528::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:02,528::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:02,531::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:02,532::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:02,532::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:02,533::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:02,534::__init__::225::IOProcess::(_processLogs) (2266) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:02,534::__init__::225::IOProcess::(_processLogs) (2266) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:02,535::__init__::225::IOProcess::(_processLogs) (2266) Building response >Thread-17::DEBUG::2014-08-22 10:07:02,536::__init__::225::IOProcess::(_processLogs) (2266) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:02,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:02,537::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:02,538::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:02,539::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:07:02,539::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:02,540::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:02,541::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:02,541::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:02,543::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:02,543::__init__::225::IOProcess::(_processLogs) (2267) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:02,544::__init__::225::IOProcess::(_processLogs) (2267) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:02,545::__init__::225::IOProcess::(_processLogs) (2267) Building response >Thread-17::DEBUG::2014-08-22 10:07:02,545::__init__::225::IOProcess::(_processLogs) (2267) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:02,546::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:02,546::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:02,548::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:02,548::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:07:02,549::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:02,550::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:02,551::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:02,551::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:02,552::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:02,553::__init__::225::IOProcess::(_processLogs) (2268) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:02,553::__init__::225::IOProcess::(_processLogs) (2268) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:02,555::__init__::225::IOProcess::(_processLogs) DEBUG|(2268) Building response >Thread-17::DEBUG::2014-08-22 10:07:02,556::__init__::225::IOProcess::(_processLogs) (2268) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:02,557::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:02,557::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:02,559::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:02,559::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:07:02,560::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:02,561::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:02,561::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:02,562::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:02,563::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:02,563::__init__::225::IOProcess::(_processLogs) (2269) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:02,564::__init__::225::IOProcess::(_processLogs) (2269) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:02,566::__init__::225::IOProcess::(_processLogs) DEBUG|(2269) Building response >Thread-17::DEBUG::2014-08-22 10:07:02,567::__init__::225::IOProcess::(_processLogs) (2269) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:02,567::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:02,568::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:07:03,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:03,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121696 s, 84.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:05,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:05,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115852 s, 88.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:07,173::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:07:07,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`55fd97f4-d9ef-46c3-be2a-96de1074650d`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:07,174::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:07:07,174::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:07,174::task::1191::Storage.TaskManager.Task::(prepare) Task=`55fd97f4-d9ef-46c3-be2a-96de1074650d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:07,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`55fd97f4-d9ef-46c3-be2a-96de1074650d`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:07,174::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:07,174::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:07,174::task::993::Storage.TaskManager.Task::(_decref) Task=`55fd97f4-d9ef-46c3-be2a-96de1074650d`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:07:07,211::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:07:07,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c5c2287-d419-40d1-96ff-620aea28a986`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:07:07,211::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:07:07,211::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3eb66474-7d33-49e6-a38e-5ebb0d11a3b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:07:07,212::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:07:07,212::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:07:07,212::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3eb66474-7d33-49e6-a38e-5ebb0d11a3b0`::Granted request >Thread-46::DEBUG::2014-08-22 10:07:07,212::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6c5c2287-d419-40d1-96ff-620aea28a986`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:07:07,212::task::993::Storage.TaskManager.Task::(_decref) Task=`6c5c2287-d419-40d1-96ff-620aea28a986`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:07:07,214::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:07,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c5c2287-d419-40d1-96ff-620aea28a986`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:07,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c5c2287-d419-40d1-96ff-620aea28a986`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:07:07,214::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:07:07,214::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:07:07,215::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:07:07,215::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:07:07,215::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:07:07,215::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:07:07,215::task::993::Storage.TaskManager.Task::(_decref) Task=`6c5c2287-d419-40d1-96ff-620aea28a986`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:07,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:07,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113831 s, 90.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:09,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f3da30b-9759-44e8-b267-8a3c402a4c3e`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:09,201::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:07:09,201::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000283875', 'lastCheck': '8.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209704', 'lastCheck': '6.6', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:09,201::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f3da30b-9759-44e8-b267-8a3c402a4c3e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000283875', 'lastCheck': '8.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000209704', 'lastCheck': '6.6', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:09,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f3da30b-9759-44e8-b267-8a3c402a4c3e`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:09,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:09,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:09,202::task::993::Storage.TaskManager.Task::(_decref) Task=`9f3da30b-9759-44e8-b267-8a3c402a4c3e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:09,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:09,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116117 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:10,500::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) (2270) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:10,501::__init__::225::IOProcess::(_processLogs) (2270) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:10,502::__init__::225::IOProcess::(_processLogs) DEBUG|(2270) Building response >Thread-21::DEBUG::2014-08-22 10:07:10,502::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:10,503::__init__::225::IOProcess::(_processLogs) (2270) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:10,509::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:07:10,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000188124 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:10,510::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:10,511::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:10,511::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) (2271) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) (2271) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) (2271) Building response >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) (2271) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:10,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:10,513::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:07:11,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:11,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130484 s, 78.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:12,567::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:12,567::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:12,567::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:12,568::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 10:07:12,568::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:12,568::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:12,574::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:12,574::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:12,575::__init__::225::IOProcess::(_processLogs) (2272) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:12,575::__init__::225::IOProcess::(_processLogs) (2272) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:12,575::__init__::225::IOProcess::(_processLogs) (2272) Building response >Thread-17::DEBUG::2014-08-22 10:07:12,575::__init__::225::IOProcess::(_processLogs) (2272) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:12,575::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:12,575::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:07:12,576::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000189254 s, 3.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:12,576::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:12,576::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:12,576::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) (2273) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) (2273) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) (2273) Building response >Thread-17::DEBUG::2014-08-22 10:07:12,577::__init__::225::IOProcess::(_processLogs) (2273) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:12,578::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:07:12,579::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:12,579::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:12,579::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:07:12,579::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:12,579::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) (2274) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) (2274) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) (2274) Building response >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) (2274) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:12,580::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:12,581::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:07:12,581::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:12,581::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:12,581::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:12,581::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:12,581::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) (2275) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) (2275) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) (2275) Building response >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) (2275) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:07:12,582::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:12,583::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:12,583::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:12,583::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) (2276) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) (2276) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) (2276) Building response >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) (2276) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:12,584::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:07:13,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:14,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125607 s, 81.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:16,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:16,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113324 s, 90.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:17,264::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:07:17,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`89fcef91-de67-4dbc-b87a-25c1601f046f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:17,264::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:07:17,265::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:17,265::task::1191::Storage.TaskManager.Task::(prepare) Task=`89fcef91-de67-4dbc-b87a-25c1601f046f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:17,265::task::595::Storage.TaskManager.Task::(_updateState) Task=`89fcef91-de67-4dbc-b87a-25c1601f046f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:17,265::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:17,265::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:17,265::task::993::Storage.TaskManager.Task::(_decref) Task=`89fcef91-de67-4dbc-b87a-25c1601f046f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:07:17,348::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:07:17,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c17b2d8-7d96-43ff-a5e8-a01cb957660c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:07:17,349::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:07:17,349::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4fae0510-1756-4546-a192-32eeaf5ef48a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:07:17,349::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:07:17,349::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:07:17,349::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4fae0510-1756-4546-a192-32eeaf5ef48a`::Granted request >Thread-46::DEBUG::2014-08-22 10:07:17,349::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0c17b2d8-7d96-43ff-a5e8-a01cb957660c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:07:17,350::task::993::Storage.TaskManager.Task::(_decref) Task=`0c17b2d8-7d96-43ff-a5e8-a01cb957660c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:07:17,352::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:17,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c17b2d8-7d96-43ff-a5e8-a01cb957660c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:17,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c17b2d8-7d96-43ff-a5e8-a01cb957660c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:07:17,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:07:17,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:07:17,352::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:07:17,352::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:07:17,352::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:07:17,352::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:07:17,353::task::993::Storage.TaskManager.Task::(_decref) Task=`0c17b2d8-7d96-43ff-a5e8-a01cb957660c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:18,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:18,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112261 s, 91.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:20,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:20,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011171 s, 91.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:20,514::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:20,514::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) (2277) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:20,515::__init__::225::IOProcess::(_processLogs) (2277) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:20,516::__init__::225::IOProcess::(_processLogs) DEBUG|(2277) Building response >Thread-21::DEBUG::2014-08-22 10:07:20,516::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:20,517::__init__::225::IOProcess::(_processLogs) (2277) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:20,523::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:20,523::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:07:20,524::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000262399 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:20,525::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:20,526::__init__::225::IOProcess::(_processLogs) (2278) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:20,526::__init__::225::IOProcess::(_processLogs) (2278) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:20,526::__init__::225::IOProcess::(_processLogs) DEBUG|(2278) Building response >Thread-17::DEBUG::2014-08-22 10:07:20,527::__init__::225::IOProcess::(_processLogs) (2278) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:20,527::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:20,527::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:07:22,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:22,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114872 s, 89.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:22,584::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:22,584::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:22,584::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:22,584::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:22,584::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:22,584::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:22,585::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:22,585::__init__::225::IOProcess::(_processLogs) (2279) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:22,585::__init__::225::IOProcess::(_processLogs) (2279) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:22,585::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2279) Building response >Thread-17::DEBUG::2014-08-22 10:07:22,586::__init__::225::IOProcess::(_processLogs) (2279) Queuing response >Thread-22::DEBUG::2014-08-22 10:07:22,586::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:22,587::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:22,593::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:07:22,593::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000261083 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:22,594::__init__::225::IOProcess::(_processLogs) (2280) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:22,595::__init__::225::IOProcess::(_processLogs) (2280) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:22,595::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2280) Building response >Thread-17::DEBUG::2014-08-22 10:07:22,596::__init__::225::IOProcess::(_processLogs) (2280) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:22,596::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:22,596::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:22,596::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:22,596::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:07:22,596::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) (2281) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) (2281) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:22,597::__init__::225::IOProcess::(_processLogs) (2281) Building response >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) (2281) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:22,598::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) (2282) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) (2282) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) (2282) Building response >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) (2282) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:22,599::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:22,600::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:22,600::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:07:22,600::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:22,600::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:22,600::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) (2283) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) (2283) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) (2283) Building response >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) (2283) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:22,601::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:07:24,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:24,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011552 s, 88.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:24,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c7dc5a2-8c3a-4e15-b740-7fdabbb6323a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:24,513::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:07:24,513::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000262399', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000261083', 'lastCheck': '1.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:24,513::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c7dc5a2-8c3a-4e15-b740-7fdabbb6323a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000262399', 'lastCheck': '4.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000261083', 'lastCheck': '1.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:24,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c7dc5a2-8c3a-4e15-b740-7fdabbb6323a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:24,513::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:24,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:24,514::task::993::Storage.TaskManager.Task::(_decref) Task=`9c7dc5a2-8c3a-4e15-b740-7fdabbb6323a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:26,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:26,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115731 s, 88.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:27,411::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:07:27,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`e061b312-086a-4278-b97b-0d213581b928`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:27,412::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:07:27,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:27,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`e061b312-086a-4278-b97b-0d213581b928`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:27,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`e061b312-086a-4278-b97b-0d213581b928`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:27,412::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:27,412::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:27,412::task::993::Storage.TaskManager.Task::(_decref) Task=`e061b312-086a-4278-b97b-0d213581b928`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:07:27,480::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:07:27,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`160c6c60-1c1b-42f3-acd8-38152c375d7f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:07:27,480::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:07:27,481::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f06c2e2c-5b1c-474f-be19-b0b30ab4cadf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:07:27,481::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:07:27,481::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:07:27,481::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f06c2e2c-5b1c-474f-be19-b0b30ab4cadf`::Granted request >Thread-46::DEBUG::2014-08-22 10:07:27,481::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`160c6c60-1c1b-42f3-acd8-38152c375d7f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:07:27,481::task::993::Storage.TaskManager.Task::(_decref) Task=`160c6c60-1c1b-42f3-acd8-38152c375d7f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:07:27,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:27,483::task::1191::Storage.TaskManager.Task::(prepare) Task=`160c6c60-1c1b-42f3-acd8-38152c375d7f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:27,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`160c6c60-1c1b-42f3-acd8-38152c375d7f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:07:27,484::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:07:27,484::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:07:27,484::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:07:27,484::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:07:27,484::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:07:27,484::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:07:27,484::task::993::Storage.TaskManager.Task::(_decref) Task=`160c6c60-1c1b-42f3-acd8-38152c375d7f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:28,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:28,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112634 s, 90.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:30,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:30,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011332 s, 90.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) (2284) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:30,528::__init__::225::IOProcess::(_processLogs) (2284) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:30,529::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2284) Building response >Thread-21::DEBUG::2014-08-22 10:07:30,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:30,530::__init__::225::IOProcess::(_processLogs) (2284) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:30,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:30,536::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:07:30,537::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00020886 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:30,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:30,537::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:30,537::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:30,537::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:30,537::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:30,538::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:30,538::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:30,538::__init__::225::IOProcess::(_processLogs) (2285) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:30,538::__init__::225::IOProcess::(_processLogs) (2285) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:30,538::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2285) Building response >Thread-17::DEBUG::2014-08-22 10:07:30,539::__init__::225::IOProcess::(_processLogs) (2285) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:30,540::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:30,540::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:07:32,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:32,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224544 s, 45.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:32,603::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:32,604::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:32,605::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:32,606::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:32,607::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:32,608::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:32,609::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:32,610::__init__::225::IOProcess::(_processLogs) (2286) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:32,611::__init__::225::IOProcess::(_processLogs) (2286) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:32,613::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2286) Building response >Thread-17::DEBUG::2014-08-22 10:07:32,614::__init__::225::IOProcess::(_processLogs) (2286) Queuing response >Thread-22::DEBUG::2014-08-22 10:07:32,614::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:32,615::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:32,622::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:07:32,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000155679 s, 3.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:32,625::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:32,626::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:32,626::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:32,628::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:32,629::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:32,630::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:32,630::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:32,631::__init__::225::IOProcess::(_processLogs) (2287) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:32,631::__init__::225::IOProcess::(_processLogs) (2287) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:32,632::__init__::225::IOProcess::(_processLogs) (2287) Building response >Thread-17::DEBUG::2014-08-22 10:07:32,632::__init__::225::IOProcess::(_processLogs) (2287) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:32,633::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:32,633::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:32,634::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:32,635::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:07:32,635::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:32,635::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:32,636::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:32,636::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:32,638::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:32,638::__init__::225::IOProcess::(_processLogs) (2288) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:32,639::__init__::225::IOProcess::(_processLogs) (2288) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:32,639::__init__::225::IOProcess::(_processLogs) (2288) Building response >Thread-17::DEBUG::2014-08-22 10:07:32,640::__init__::225::IOProcess::(_processLogs) (2288) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:32,640::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:32,641::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:32,642::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:32,642::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:07:32,643::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:32,643::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:32,643::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:32,644::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:32,644::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:32,645::__init__::225::IOProcess::(_processLogs) (2289) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:32,646::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:32,647::__init__::225::IOProcess::(_processLogs) (2289) Building response >Thread-17::DEBUG::2014-08-22 10:07:32,647::__init__::225::IOProcess::(_processLogs) (2289) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:32,648::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:32,648::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:32,649::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:32,649::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:07:32,650::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:32,650::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:32,651::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:32,652::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:32,653::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:32,654::__init__::225::IOProcess::(_processLogs) (2290) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:32,654::__init__::225::IOProcess::(_processLogs) (2290) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:32,655::__init__::225::IOProcess::(_processLogs) (2290) Building response >Thread-17::DEBUG::2014-08-22 10:07:32,655::__init__::225::IOProcess::(_processLogs) (2290) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:32,656::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:32,656::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:07:34,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:34,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119145 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:36,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:36,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117136 s, 87.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:37,538::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:07:37,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`e53cf185-8675-4a7d-849d-38f1fe4935da`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:37,539::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:07:37,539::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:37,539::task::1191::Storage.TaskManager.Task::(prepare) Task=`e53cf185-8675-4a7d-849d-38f1fe4935da`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:37,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`e53cf185-8675-4a7d-849d-38f1fe4935da`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:37,539::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:37,540::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:37,540::task::993::Storage.TaskManager.Task::(_decref) Task=`e53cf185-8675-4a7d-849d-38f1fe4935da`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:07:37,564::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:07:37,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`43416dc9-ec03-4c6b-99d4-21d978e6b85b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:07:37,564::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:07:37,565::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a1cac223-6602-4c8d-a146-077e415a74f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:07:37,565::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:07:37,565::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:07:37,565::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a1cac223-6602-4c8d-a146-077e415a74f4`::Granted request >Thread-46::DEBUG::2014-08-22 10:07:37,565::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`43416dc9-ec03-4c6b-99d4-21d978e6b85b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:07:37,565::task::993::Storage.TaskManager.Task::(_decref) Task=`43416dc9-ec03-4c6b-99d4-21d978e6b85b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:07:37,567::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:37,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`43416dc9-ec03-4c6b-99d4-21d978e6b85b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:37,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`43416dc9-ec03-4c6b-99d4-21d978e6b85b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:07:37,567::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:07:37,568::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:07:37,568::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:07:37,568::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:07:37,568::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:07:37,568::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:07:37,568::task::993::Storage.TaskManager.Task::(_decref) Task=`43416dc9-ec03-4c6b-99d4-21d978e6b85b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:38,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:38,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113773 s, 90.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:39,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b4ef242-565a-4975-a6e3-bb52eba0579a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:39,679::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:07:39,679::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00020886', 'lastCheck': '9.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000155679', 'lastCheck': '7.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:39,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b4ef242-565a-4975-a6e3-bb52eba0579a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00020886', 'lastCheck': '9.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000155679', 'lastCheck': '7.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:39,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b4ef242-565a-4975-a6e3-bb52eba0579a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:39,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:39,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:39,680::task::993::Storage.TaskManager.Task::(_decref) Task=`1b4ef242-565a-4975-a6e3-bb52eba0579a`::ref 0 aborting False >Thread-21::DEBUG::2014-08-22 10:07:40,540::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 10:07:40,541::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,541::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 10:07:40,541::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,542::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,542::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,542::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,542::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,545::__init__::225::IOProcess::(_processLogs) (2291) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:07:40,546::__init__::225::IOProcess::(_processLogs) (2291) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:07:40,547::__init__::225::IOProcess::(_processLogs) (2291) Building response >Thread-21::DEBUG::2014-08-22 10:07:40,547::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 10:07:40,548::__init__::225::IOProcess::(_processLogs) (2291) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,548::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:40,548::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 10:07:40,548::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,548::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 10:07:40,548::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) (2292) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) (2292) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:07:40,549::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|(2292) Building response >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) (2292) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,550::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,551::__init__::225::IOProcess::(_processLogs) (2293) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 10:07:40,551::__init__::225::IOProcess::(_processLogs) (2293) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 10:07:40,577::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|(2293) Building response >Thread-17::DEBUG::2014-08-22 10:07:40,577::__init__::225::IOProcess::(_processLogs) (2293) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,578::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 10:07:40,578::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 10:07:40,578::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) (2294) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 10:07:40,579::__init__::225::IOProcess::(_processLogs) (2294) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 10:07:40,580::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|(2294) Building response >Thread-21::DEBUG::2014-08-22 10:07:40,580::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 10:07:40,580::__init__::225::IOProcess::(_processLogs) (2294) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,582::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:40,582::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 10:07:40,582::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,582::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 10:07:40,582::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) (2295) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) (2295) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) (2295) Building response >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) (2295) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,583::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) (2296) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:07:40,584::__init__::225::IOProcess::(_processLogs) (2296) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:07:40,585::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|(2296) Building response >Thread-21::DEBUG::2014-08-22 10:07:40,585::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 10:07:40,585::__init__::225::IOProcess::(_processLogs) (2296) Queuing response >Thread-21::INFO::2014-08-22 10:07:40,585::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 10:07:40,586::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 10:07:40,586::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 10:07:40,587::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 10:07:40,587::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,587::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:40,587::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,588::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:07:40,589::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:40,589::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,595::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:07:40,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000225366 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:40,596::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) (2297) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) (2297) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) (2297) Building response >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) (2297) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) (2289) Got request for method 'access'DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:40,598::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) (2298) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) (2298) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:40,599::__init__::225::IOProcess::(_processLogs) (2298) Building response >Thread-17::DEBUG::2014-08-22 10:07:40,600::__init__::225::IOProcess::(_processLogs) (2298) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:40,601::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:40,601::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:07:40,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:40,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112998 s, 90.6 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-08-22 10:07:42,653::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) (2299) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:07:42,655::__init__::225::IOProcess::(_processLogs) (2299) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) (2299) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:42,656::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,657::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,657::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,657::__init__::225::IOProcess::(_processLogs) (2300) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:07:42,657::__init__::225::IOProcess::(_processLogs) (2300) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:07:42,657::__init__::225::IOProcess::(_processLogs) (2300) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,658::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,659::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 10:07:42,661::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) (2301) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 10:07:42,662::__init__::225::IOProcess::(_processLogs) (2301) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 10:07:42,677::__init__::225::IOProcess::(_processLogs) (2301) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,677::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-22::DEBUG::2014-08-22 10:07:42,677::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 10:07:42,678::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 10:07:42,678::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,678::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,678::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,678::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,679::__init__::225::IOProcess::(_processLogs) (2302) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 10:07:42,679::__init__::225::IOProcess::(_processLogs) (2302) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 10:07:42,679::__init__::225::IOProcess::(_processLogs) (2302) Queuing response >Thread-22::DEBUG::2014-08-22 10:07:42,679::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 10:07:42,680::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,680::__init__::225::IOProcess::(_processLogs) Sending response sized 832 >Thread-17::DEBUG::2014-08-22 10:07:42,681::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 10:07:42,681::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,682::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,682::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,682::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,682::__init__::225::IOProcess::(_processLogs) (2303) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:42,682::__init__::225::IOProcess::(_processLogs) (2303) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:42,682::__init__::225::IOProcess::(_processLogs) (2303) Building response >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) (2303) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) (2304) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:07:42,683::__init__::225::IOProcess::(_processLogs) (2304) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:07:42,690::__init__::225::IOProcess::(_processLogs) (2304) Queuing response >Thread-22::DEBUG::2014-08-22 10:07:42,691::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 10:07:42,691::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 10:07:42,691::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 10:07:42,691::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-22::INFO::2014-08-22 10:07:42,692::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 10:07:42,693::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:42,693::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,693::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,694::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,694::__init__::225::IOProcess::(_processLogs) (2305) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:42,694::__init__::225::IOProcess::(_processLogs) (2305) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:42,694::__init__::225::IOProcess::(_processLogs) (2305) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,695::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:07:42,695::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:42,696::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:07:42,702::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000200101 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:42,703::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:42,703::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,703::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) (2306) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) (2306) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) (2306) Building response >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) (2306) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,704::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,705::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:42,705::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:07:42,705::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,705::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,705::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) (2307) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) (2307) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) (2307) Building response >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) (2307) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:07:42,706::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) (2308) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) (2308) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) (2308) Building response >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) (2308) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,707::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:42,708::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:07:42,708::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:42,708::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:42,708::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) (2309) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) (2309) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) (2309) Building response >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) (2309) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:42,709::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:07:42,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:42,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113348 s, 90.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:44,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:44,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116038 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:46,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:46,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114363 s, 89.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:47,618::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:07:47,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`9218cd7b-0067-401e-80e3-de93260f2442`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:47,618::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:07:47,619::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:47,619::task::1191::Storage.TaskManager.Task::(prepare) Task=`9218cd7b-0067-401e-80e3-de93260f2442`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:47,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`9218cd7b-0067-401e-80e3-de93260f2442`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:47,619::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:47,619::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:47,619::task::993::Storage.TaskManager.Task::(_decref) Task=`9218cd7b-0067-401e-80e3-de93260f2442`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:07:47,651::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:07:47,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`30ea7090-cefb-42b9-84d4-a334e02a6c1c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:07:47,651::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:07:47,652::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`89fc74f1-f21f-4df3-bf0c-e961bac298d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:07:47,652::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:07:47,652::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:07:47,652::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`89fc74f1-f21f-4df3-bf0c-e961bac298d5`::Granted request >Thread-46::DEBUG::2014-08-22 10:07:47,652::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`30ea7090-cefb-42b9-84d4-a334e02a6c1c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:07:47,652::task::993::Storage.TaskManager.Task::(_decref) Task=`30ea7090-cefb-42b9-84d4-a334e02a6c1c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:07:47,654::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:47,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`30ea7090-cefb-42b9-84d4-a334e02a6c1c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:47,655::task::595::Storage.TaskManager.Task::(_updateState) Task=`30ea7090-cefb-42b9-84d4-a334e02a6c1c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:07:47,655::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:07:47,655::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:07:47,655::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:07:47,655::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:07:47,655::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:07:47,655::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:07:47,655::task::993::Storage.TaskManager.Task::(_decref) Task=`30ea7090-cefb-42b9-84d4-a334e02a6c1c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:48,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:49,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114465 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:50,601::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:50,601::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:50,601::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:50,601::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:50,602::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:50,602::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:50,602::__init__::225::IOProcess::(_processLogs) (2310) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:50,602::__init__::225::IOProcess::(_processLogs) (2310) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:50,602::__init__::225::IOProcess::(_processLogs) DEBUG|(2310) Building response >Thread-21::DEBUG::2014-08-22 10:07:50,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:50,603::__init__::225::IOProcess::(_processLogs) (2310) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:50,610::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:07:50,611::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000261041 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:50,611::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:50,612::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:07:50,612::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:07:50,613::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:50,614::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:50,614::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:50,614::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:50,614::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:50,614::__init__::225::IOProcess::(_processLogs) (2311) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:50,614::__init__::225::IOProcess::(_processLogs) (2311) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:50,615::__init__::225::IOProcess::(_processLogs) (2311) Building response >Thread-17::DEBUG::2014-08-22 10:07:50,615::__init__::225::IOProcess::(_processLogs) (2311) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:50,615::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:50,615::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:07:51,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:51,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117213 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:52,709::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:52,709::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:52,711::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:52,711::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:52,711::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:52,711::__init__::225::IOProcess::(_processLogs) (2312) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:52,711::__init__::225::IOProcess::(_processLogs) (2312) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:52,712::__init__::225::IOProcess::(_processLogs) (2312) Queuing response >Thread-22::DEBUG::2014-08-22 10:07:52,712::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:07:52,712::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:52,719::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:07:52,720::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000229935 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:07:52,720::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:07:52,720::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:07:52,720::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:52,721::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:52,721::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:52,721::__init__::225::IOProcess::(_processLogs) (2313) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:07:52,721::__init__::225::IOProcess::(_processLogs) (2313) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:07:52,722::__init__::225::IOProcess::(_processLogs) (2313) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:52,722::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:52,722::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:07:52,723::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:07:52,723::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:52,723::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:07:52,723::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:52,723::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:52,723::__init__::225::IOProcess::(_processLogs) (2314) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) (2314) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) (2314) Building response >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) (2314) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:07:52,724::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) (2315) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) (2315) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) (2315) Building response >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) (2315) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:52,725::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:07:52,726::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:07:52,726::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:07:52,726::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) (2316) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) (2316) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) (2316) Building response >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) (2316) Queuing response >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:07:52,727::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:07:53,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:53,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118249 s, 86.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:54,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`93647400-9146-47d5-b268-7f8ea9f89705`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:54,980::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:07:54,980::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000261041', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000229935', 'lastCheck': '2.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:54,980::task::1191::Storage.TaskManager.Task::(prepare) Task=`93647400-9146-47d5-b268-7f8ea9f89705`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000261041', 'lastCheck': '4.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000229935', 'lastCheck': '2.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:07:54,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`93647400-9146-47d5-b268-7f8ea9f89705`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:54,980::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:54,980::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:54,980::task::993::Storage.TaskManager.Task::(_decref) Task=`93647400-9146-47d5-b268-7f8ea9f89705`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:55,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:55,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114159 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:07:57,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:57,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116247 s, 88.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:07:57,705::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [5a785a8f] >Thread-18::DEBUG::2014-08-22 10:07:57,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ca1394a-d7e1-4fd2-9836-94c1bb82ef3b`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:07:57,705::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:07:57,706::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:57,706::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ca1394a-d7e1-4fd2-9836-94c1bb82ef3b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:07:57,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ca1394a-d7e1-4fd2-9836-94c1bb82ef3b`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:07:57,706::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:07:57,706::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:07:57,706::task::993::Storage.TaskManager.Task::(_decref) Task=`6ca1394a-d7e1-4fd2-9836-94c1bb82ef3b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:07:57,737::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [5a785a8f] >Thread-46::DEBUG::2014-08-22 10:07:57,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3898d51-6a8e-4a24-8258-9c046b9381f5`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:07:57,738::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:07:57,738::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`043fa99a-cb81-458f-b4bc-49dc1e5c2864`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:07:57,738::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:07:57,738::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:07:57,738::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`043fa99a-cb81-458f-b4bc-49dc1e5c2864`::Granted request >Thread-46::DEBUG::2014-08-22 10:07:57,739::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a3898d51-6a8e-4a24-8258-9c046b9381f5`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:07:57,739::task::993::Storage.TaskManager.Task::(_decref) Task=`a3898d51-6a8e-4a24-8258-9c046b9381f5`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:07:57,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:57,741::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3898d51-6a8e-4a24-8258-9c046b9381f5`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:07:57,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3898d51-6a8e-4a24-8258-9c046b9381f5`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:07:57,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:07:57,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:07:57,741::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:07:57,741::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:07:57,741::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:07:57,742::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:07:57,742::task::993::Storage.TaskManager.Task::(_decref) Task=`a3898d51-6a8e-4a24-8258-9c046b9381f5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:07:59,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:07:59,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113306 s, 90.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:00,615::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:00,615::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:00,615::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:00,616::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:08:00,616::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:00,617::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:00,623::__init__::225::IOProcess::(_processLogs) (2317) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 10:08:00,624::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000192522 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:00,625::__init__::225::IOProcess::(_processLogs) (2317) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:00,625::__init__::225::IOProcess::(_processLogs) (2317) Building response >Thread-17::DEBUG::2014-08-22 10:08:00,625::__init__::225::IOProcess::(_processLogs) (2317) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:00,625::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) (2318) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:00,626::__init__::225::IOProcess::(_processLogs) (2318) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:00,627::__init__::225::IOProcess::(_processLogs) (2318) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:00,628::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:00,628::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:08:01,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:01,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118954 s, 86.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:02,728::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:02,728::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:02,729::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:02,729::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:02,731::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:08:02,732::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:02,732::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:02,739::__init__::225::IOProcess::(_processLogs) (2319) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:02,741::__init__::225::IOProcess::(_processLogs) (2319) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 10:08:02,741::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000198501 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:02,742::__init__::225::IOProcess::(_processLogs) (2319) Building response >Thread-17::DEBUG::2014-08-22 10:08:02,743::__init__::225::IOProcess::(_processLogs) (2319) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:02,744::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:02,744::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:02,745::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:02,746::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:02,746::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:02,747::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:02,747::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:02,749::__init__::225::IOProcess::(_processLogs) (2320) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:02,750::__init__::225::IOProcess::(_processLogs) (2320) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:02,751::__init__::225::IOProcess::(_processLogs) (2320) Building response >Thread-17::DEBUG::2014-08-22 10:08:02,751::__init__::225::IOProcess::(_processLogs) (2320) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:02,751::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:02,752::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:02,753::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:08:02,753::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:02,754::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:02,754::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:02,755::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:02,756::__init__::225::IOProcess::(_processLogs) (2321) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:02,757::__init__::225::IOProcess::(_processLogs) (2321) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:02,757::__init__::225::IOProcess::(_processLogs) (2321) Building response >Thread-17::DEBUG::2014-08-22 10:08:02,758::__init__::225::IOProcess::(_processLogs) (2321) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:02,758::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:02,758::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:02,759::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:08:02,760::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:02,760::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:02,761::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:02,761::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:02,762::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:02,763::__init__::225::IOProcess::(_processLogs) (2322) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:02,763::__init__::225::IOProcess::(_processLogs) (2322) Building response >Thread-17::DEBUG::2014-08-22 10:08:02,764::__init__::225::IOProcess::(_processLogs) (2322) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:02,764::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:02,765::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:02,766::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:08:02,766::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:02,767::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:02,767::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:02,768::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:02,769::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:02,770::__init__::225::IOProcess::(_processLogs) (2323) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:02,770::__init__::225::IOProcess::(_processLogs) (2323) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:02,771::__init__::225::IOProcess::(_processLogs) (2323) Building response >Thread-17::DEBUG::2014-08-22 10:08:02,771::__init__::225::IOProcess::(_processLogs) (2323) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:02,772::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:02,772::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:08:03,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:03,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152115 s, 67.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:05,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:05,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114118 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:07,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:07,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115907 s, 88.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:07,801::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:08:07,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`74aa3ac1-1e7b-4022-a7b1-f04250b68388`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:07,802::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:08:07,802::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:07,802::task::1191::Storage.TaskManager.Task::(prepare) Task=`74aa3ac1-1e7b-4022-a7b1-f04250b68388`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:07,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`74aa3ac1-1e7b-4022-a7b1-f04250b68388`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:07,803::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:07,803::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:07,803::task::993::Storage.TaskManager.Task::(_decref) Task=`74aa3ac1-1e7b-4022-a7b1-f04250b68388`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:08:07,835::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:08:07,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`b55218c5-92e9-4141-9296-c682c87425bb`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:08:07,835::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:08:07,835::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ef58bf55-35b4-48f5-aa84-9047cf6e8c54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:08:07,835::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:08:07,836::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:08:07,836::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`ef58bf55-35b4-48f5-aa84-9047cf6e8c54`::Granted request >Thread-46::DEBUG::2014-08-22 10:08:07,836::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b55218c5-92e9-4141-9296-c682c87425bb`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:08:07,836::task::993::Storage.TaskManager.Task::(_decref) Task=`b55218c5-92e9-4141-9296-c682c87425bb`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:08:07,838::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:07,838::task::1191::Storage.TaskManager.Task::(prepare) Task=`b55218c5-92e9-4141-9296-c682c87425bb`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:07,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`b55218c5-92e9-4141-9296-c682c87425bb`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:08:07,838::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:08:07,838::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:08:07,839::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:08:07,839::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:08:07,839::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:08:07,839::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:08:07,839::task::993::Storage.TaskManager.Task::(_decref) Task=`b55218c5-92e9-4141-9296-c682c87425bb`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:09,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:09,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118039 s, 86.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:10,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`10b96ba7-163e-4ace-9224-c3c2a1a45c1f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:10,260::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:08:10,260::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000192522', 'lastCheck': '9.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198501', 'lastCheck': '7.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:10,260::task::1191::Storage.TaskManager.Task::(prepare) Task=`10b96ba7-163e-4ace-9224-c3c2a1a45c1f`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000192522', 'lastCheck': '9.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000198501', 'lastCheck': '7.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:10,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`10b96ba7-163e-4ace-9224-c3c2a1a45c1f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:10,260::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:10,260::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:10,260::task::993::Storage.TaskManager.Task::(_decref) Task=`10b96ba7-163e-4ace-9224-c3c2a1a45c1f`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:08:10,628::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:10,628::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:10,628::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:10,629::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:10,629::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:10,629::__init__::225::IOProcess::(_processLogs) (2324) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:10,629::__init__::225::IOProcess::(_processLogs) (2324) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:10,629::__init__::225::IOProcess::(_processLogs) (2324) Queuing response >Thread-21::DEBUG::2014-08-22 10:08:10,630::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:10,630::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:10,637::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:08:10,638::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000218478 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) (2325) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:10,639::__init__::225::IOProcess::(_processLogs) (2325) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:10,640::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2325) Building response >Thread-17::DEBUG::2014-08-22 10:08:10,641::__init__::225::IOProcess::(_processLogs) (2325) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:10,641::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:10,641::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:08:11,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:11,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119634 s, 85.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:12,770::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:12,770::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:12,770::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:12,771::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 10:08:12,772::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:12,772::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:12,778::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:08:12,779::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000209242 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:12,780::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) (2326) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) (2326) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) (2326) Building response >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) (2326) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:12,781::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:12,782::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:12,782::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:12,782::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:12,782::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:12,782::__init__::225::IOProcess::(_processLogs) (2327) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:12,782::__init__::225::IOProcess::(_processLogs) (2327) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:12,783::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2327) Building response >Thread-17::DEBUG::2014-08-22 10:08:12,783::__init__::225::IOProcess::(_processLogs) (2327) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:12,783::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:12,783::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:12,784::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:12,784::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:08:12,784::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:12,784::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:12,784::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) (2328) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) (2328) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) (2328) Building response >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) (2328) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:08:12,785::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) (2329) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) (2329) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:12,786::__init__::225::IOProcess::(_processLogs) (2329) Building response >Thread-17::DEBUG::2014-08-22 10:08:12,787::__init__::225::IOProcess::(_processLogs) (2329) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:12,787::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:12,787::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:12,787::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:12,787::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:08:12,787::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:12,788::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:12,788::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:12,788::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:12,788::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:12,788::__init__::225::IOProcess::(_processLogs) (2330) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:12,788::__init__::225::IOProcess::(_processLogs) (2330) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:12,789::__init__::225::IOProcess::(_processLogs) (2330) Building response >Thread-17::DEBUG::2014-08-22 10:08:12,789::__init__::225::IOProcess::(_processLogs) (2330) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:12,789::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:12,789::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:08:13,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:13,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011663 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:15,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:15,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200286 s, 51.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:17,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:17,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115373 s, 88.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:17,888::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:08:17,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`967eadee-7545-4440-945b-4c921d165c3b`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:17,889::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:08:17,889::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:17,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`967eadee-7545-4440-945b-4c921d165c3b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:17,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`967eadee-7545-4440-945b-4c921d165c3b`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:17,889::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:17,889::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:17,889::task::993::Storage.TaskManager.Task::(_decref) Task=`967eadee-7545-4440-945b-4c921d165c3b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:08:17,930::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:08:17,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6a53788-543c-4e8e-97d6-47fed739c37d`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:08:17,930::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:08:17,930::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`30228210-eb60-4f92-8943-d4b1d66a11fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:08:17,930::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:08:17,930::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:08:17,931::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`30228210-eb60-4f92-8943-d4b1d66a11fa`::Granted request >Thread-46::DEBUG::2014-08-22 10:08:17,931::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f6a53788-543c-4e8e-97d6-47fed739c37d`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:08:17,931::task::993::Storage.TaskManager.Task::(_decref) Task=`f6a53788-543c-4e8e-97d6-47fed739c37d`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:08:17,933::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:17,933::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6a53788-543c-4e8e-97d6-47fed739c37d`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:17,933::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6a53788-543c-4e8e-97d6-47fed739c37d`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:08:17,933::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:08:17,933::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:08:17,933::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:08:17,934::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:08:17,934::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:08:17,934::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:08:17,934::task::993::Storage.TaskManager.Task::(_decref) Task=`f6a53788-543c-4e8e-97d6-47fed739c37d`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:19,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:19,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139955 s, 73.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:20,641::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:20,641::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) (2331) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:20,642::__init__::225::IOProcess::(_processLogs) (2331) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:20,643::__init__::225::IOProcess::(_processLogs) (2331) Queuing response >Thread-21::DEBUG::2014-08-22 10:08:20,643::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:20,644::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:20,650::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:08:20,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000232967 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) (2332) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:20,652::__init__::225::IOProcess::(_processLogs) (2332) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:20,653::__init__::225::IOProcess::(_processLogs) (2332) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:20,654::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:20,654::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:08:21,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:21,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115258 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) (2333) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:22,789::__init__::225::IOProcess::(_processLogs) (2333) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:22,790::__init__::225::IOProcess::(_processLogs) (2333) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:22,790::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:08:22,790::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:22,791::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:08:22,797::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000204173 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) (2334) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) (2334) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) (2334) Building response >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) (2334) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:22,798::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:22,799::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:08:22,799::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) (2335) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) (2335) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) (2335) Building response >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) (2335) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:22,800::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:22,801::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:22,801::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:08:22,801::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:22,801::__init__::225::IOProcess::(_processLogs) DEBUGWaiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) (2336) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) (2336) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) (2336) Building response >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) (2336) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:08:22,802::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:22,803::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:22,803::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:22,803::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:22,804::__init__::225::IOProcess::(_processLogs) (2337) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:22,804::__init__::225::IOProcess::(_processLogs) (2337) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:22,804::__init__::225::IOProcess::(_processLogs) (2337) Building response >Thread-17::DEBUG::2014-08-22 10:08:22,804::__init__::225::IOProcess::(_processLogs) (2337) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:22,804::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:22,804::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:08:23,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:23,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011657 s, 87.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:25,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`369f1b98-4653-4007-8931-ec4c78631e69`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:25,500::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:08:25,500::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232967', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000204173', 'lastCheck': '2.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:25,500::task::1191::Storage.TaskManager.Task::(prepare) Task=`369f1b98-4653-4007-8931-ec4c78631e69`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000232967', 'lastCheck': '4.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000204173', 'lastCheck': '2.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:25,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`369f1b98-4653-4007-8931-ec4c78631e69`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:25,500::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:25,500::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:25,500::task::993::Storage.TaskManager.Task::(_decref) Task=`369f1b98-4653-4007-8931-ec4c78631e69`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:25,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:25,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117341 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:27,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:27,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011539 s, 88.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:28,033::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-18::DEBUG::2014-08-22 10:08:28,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`778f006b-cd25-48d6-86fa-bcd8b9903524`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:28,034::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:08:28,034::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:28,034::task::1191::Storage.TaskManager.Task::(prepare) Task=`778f006b-cd25-48d6-86fa-bcd8b9903524`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:28,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`778f006b-cd25-48d6-86fa-bcd8b9903524`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:28,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:28,034::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:28,035::task::993::Storage.TaskManager.Task::(_decref) Task=`778f006b-cd25-48d6-86fa-bcd8b9903524`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:08:28,083::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [6644c7b1] >Thread-46::DEBUG::2014-08-22 10:08:28,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdb65c1b-e0f9-4c16-bef6-80dec22ba53f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:08:28,083::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:08:28,084::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fc3ef73a-9af4-440a-963a-233e97e937dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:08:28,084::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:08:28,084::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:08:28,084::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`fc3ef73a-9af4-440a-963a-233e97e937dd`::Granted request >Thread-46::DEBUG::2014-08-22 10:08:28,085::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cdb65c1b-e0f9-4c16-bef6-80dec22ba53f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:08:28,086::task::993::Storage.TaskManager.Task::(_decref) Task=`cdb65c1b-e0f9-4c16-bef6-80dec22ba53f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:08:28,087::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:28,088::task::1191::Storage.TaskManager.Task::(prepare) Task=`cdb65c1b-e0f9-4c16-bef6-80dec22ba53f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:28,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdb65c1b-e0f9-4c16-bef6-80dec22ba53f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:08:28,088::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:08:28,088::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:08:28,088::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:08:28,088::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:08:28,088::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:08:28,088::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:08:28,088::task::993::Storage.TaskManager.Task::(_decref) Task=`cdb65c1b-e0f9-4c16-bef6-80dec22ba53f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:29,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:30,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117051 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:30,654::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:30,655::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:30,655::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:30,655::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:30,655::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:30,655::__init__::225::IOProcess::(_processLogs) (2338) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:30,655::__init__::225::IOProcess::(_processLogs) (2338) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:30,656::__init__::225::IOProcess::(_processLogs) (2338) Queuing response >Thread-21::DEBUG::2014-08-22 10:08:30,657::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:30,657::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:30,664::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:08:30,665::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000182931 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:30,665::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:30,665::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:30,665::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:30,665::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:30,666::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:30,666::__init__::225::IOProcess::(_processLogs) (2339) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:30,666::__init__::225::IOProcess::(_processLogs) (2339) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:30,666::__init__::225::IOProcess::(_processLogs) (2339) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:30,667::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:30,667::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:08:32,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:32,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173631 s, 59.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:32,805::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:32,805::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:32,806::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:32,806::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:32,807::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:32,809::__init__::225::IOProcess::(_processLogs) (2340) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:32,810::__init__::225::IOProcess::(_processLogs) (2340) Got request for method 'statvfs' >Thread-22::DEBUG::2014-08-22 10:08:32,810::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:32,816::__init__::225::IOProcess::(_processLogs) (2340) Building response >Thread-17::DEBUG::2014-08-22 10:08:32,818::__init__::225::IOProcess::(_processLogs) (2340) Queuing response >Thread-22::DEBUG::2014-08-22 10:08:32,819::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00018352 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:32,819::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:32,821::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:32,822::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:32,822::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:32,824::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:32,825::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:32,826::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:32,826::__init__::225::IOProcess::(_processLogs) (2341) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:32,827::__init__::225::IOProcess::(_processLogs) (2341) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:32,827::__init__::225::IOProcess::(_processLogs) (2341) Building response >Thread-17::DEBUG::2014-08-22 10:08:32,828::__init__::225::IOProcess::(_processLogs) (2341) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:32,828::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:32,829::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:32,830::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:08:32,830::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:32,831::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:32,831::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:32,832::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:32,833::__init__::225::IOProcess::(_processLogs) (2342) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:32,833::__init__::225::IOProcess::(_processLogs) (2342) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:32,834::__init__::225::IOProcess::(_processLogs) (2342) Building response >Thread-17::DEBUG::2014-08-22 10:08:32,834::__init__::225::IOProcess::(_processLogs) (2342) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:32,835::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:32,835::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:32,836::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:08:32,837::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:32,837::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:32,838::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:32,838::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:32,839::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:32,839::__init__::225::IOProcess::(_processLogs) (2343) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:32,840::__init__::225::IOProcess::(_processLogs) (2343) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:32,841::__init__::225::IOProcess::(_processLogs) (2343) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:32,841::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:32,842::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:32,843::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:08:32,843::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:32,844::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:32,844::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:32,845::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:32,846::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:32,847::__init__::225::IOProcess::(_processLogs) (2344) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:32,847::__init__::225::IOProcess::(_processLogs) (2344) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:32,848::__init__::225::IOProcess::(_processLogs) (2344) Building response >Thread-17::DEBUG::2014-08-22 10:08:32,848::__init__::225::IOProcess::(_processLogs) (2344) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:32,849::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:32,849::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:08:34,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:34,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123454 s, 82.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:36,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:36,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115892 s, 88.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:38,144::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:08:38,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f2f5fa3-9dcf-49ea-9819-bdf544730486`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:38,145::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:08:38,145::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:38,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f2f5fa3-9dcf-49ea-9819-bdf544730486`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:38,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f2f5fa3-9dcf-49ea-9819-bdf544730486`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:38,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:38,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:38,145::task::993::Storage.TaskManager.Task::(_decref) Task=`9f2f5fa3-9dcf-49ea-9819-bdf544730486`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:08:38,179::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:08:38,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`70d3ba6b-2faf-46ab-bef3-85ab97b770d2`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:08:38,179::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:08:38,179::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6367ac1c-73ac-4dec-a9a8-a9646cc76e9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:08:38,179::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:08:38,180::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:08:38,180::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6367ac1c-73ac-4dec-a9a8-a9646cc76e9c`::Granted request >Thread-46::DEBUG::2014-08-22 10:08:38,180::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`70d3ba6b-2faf-46ab-bef3-85ab97b770d2`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:08:38,180::task::993::Storage.TaskManager.Task::(_decref) Task=`70d3ba6b-2faf-46ab-bef3-85ab97b770d2`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:08:38,182::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:38,182::task::1191::Storage.TaskManager.Task::(prepare) Task=`70d3ba6b-2faf-46ab-bef3-85ab97b770d2`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:38,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`70d3ba6b-2faf-46ab-bef3-85ab97b770d2`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:08:38,182::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:08:38,182::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:08:38,182::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:08:38,183::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:08:38,183::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:08:38,183::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:08:38,183::task::993::Storage.TaskManager.Task::(_decref) Task=`70d3ba6b-2faf-46ab-bef3-85ab97b770d2`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:38,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:38,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114457 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:40,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:40,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113703 s, 90.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:40,668::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:40,668::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:40,668::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:40,669::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:08:40,669::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:40,669::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:40,676::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:08:40,677::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000218672 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:40,677::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:40,678::__init__::225::IOProcess::(_processLogs) (2345) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:40,678::__init__::225::IOProcess::(_processLogs) (2345) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:40,678::__init__::225::IOProcess::(_processLogs) (2345) Building response >Thread-17::DEBUG::2014-08-22 10:08:40,678::__init__::225::IOProcess::(_processLogs) (2345) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:40,678::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:40,679::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:40,680::__init__::225::IOProcess::(_processLogs) (2346) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:40,680::__init__::225::IOProcess::(_processLogs) (2346) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:40,680::__init__::225::IOProcess::(_processLogs) DEBUG|(2346) Building response >Thread-17::DEBUG::2014-08-22 10:08:40,681::__init__::225::IOProcess::(_processLogs) (2346) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:40,681::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:40,681::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:08:40,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae5ca9ca-848c-4b11-909c-1a9ee01a7cc5`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:40,684::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:08:40,684::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000218672', 'lastCheck': '0.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00018352', 'lastCheck': '7.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:40,684::task::1191::Storage.TaskManager.Task::(prepare) Task=`ae5ca9ca-848c-4b11-909c-1a9ee01a7cc5`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000218672', 'lastCheck': '0.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00018352', 'lastCheck': '7.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:40,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae5ca9ca-848c-4b11-909c-1a9ee01a7cc5`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:40,684::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:40,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:40,685::task::993::Storage.TaskManager.Task::(_decref) Task=`ae5ca9ca-848c-4b11-909c-1a9ee01a7cc5`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:42,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:42,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117828 s, 86.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:42,847::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) (2347) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) (2347) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) (2347) Building response >Thread-17::DEBUG::2014-08-22 10:08:42,848::__init__::225::IOProcess::(_processLogs) (2347) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:42,849::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:42,849::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:08:42,849::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:08:42,855::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000236277 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:42,856::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:42,856::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:42,856::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:42,856::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:42,856::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) (2348) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) (2348) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) (2348) Building response >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) (2348) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:42,857::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:42,858::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:42,858::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:42,859::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:08:42,859::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:42,859::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:42,859::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:42,859::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) (2349) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) (2349) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) (2349) Building response >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) (2349) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:08:42,860::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) Waiting for next request...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) (2350) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) (2350) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) (2350) Building response >Thread-17::DEBUG::2014-08-22 10:08:42,861::__init__::225::IOProcess::(_processLogs) (2350) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:42,862::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:42,862::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:42,862::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:42,862::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:08:42,862::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:42,862::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:42,863::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:42,863::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:42,863::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:42,863::__init__::225::IOProcess::(_processLogs) (2351) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:42,863::__init__::225::IOProcess::(_processLogs) (2351) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:42,863::__init__::225::IOProcess::(_processLogs) (2351) Building response >Thread-17::DEBUG::2014-08-22 10:08:42,864::__init__::225::IOProcess::(_processLogs) (2351) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:42,864::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:42,864::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:08:44,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:44,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117817 s, 86.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:46,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:46,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113251 s, 90.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:48,233::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:08:48,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`c772c959-f534-4de2-8cfd-587c7e70fb33`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:48,234::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:08:48,234::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:48,234::task::1191::Storage.TaskManager.Task::(prepare) Task=`c772c959-f534-4de2-8cfd-587c7e70fb33`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:48,234::task::595::Storage.TaskManager.Task::(_updateState) Task=`c772c959-f534-4de2-8cfd-587c7e70fb33`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:48,234::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:48,234::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:48,234::task::993::Storage.TaskManager.Task::(_decref) Task=`c772c959-f534-4de2-8cfd-587c7e70fb33`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:08:48,275::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:08:48,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e661a46-3078-400d-b845-2e2397852b0c`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:08:48,275::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:08:48,275::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f0cd0368-bb91-4694-ae41-8e33b52e493d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:08:48,275::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:08:48,275::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:08:48,275::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f0cd0368-bb91-4694-ae41-8e33b52e493d`::Granted request >Thread-46::DEBUG::2014-08-22 10:08:48,276::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9e661a46-3078-400d-b845-2e2397852b0c`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:08:48,276::task::993::Storage.TaskManager.Task::(_decref) Task=`9e661a46-3078-400d-b845-2e2397852b0c`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:08:48,278::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:48,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e661a46-3078-400d-b845-2e2397852b0c`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:48,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e661a46-3078-400d-b845-2e2397852b0c`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:08:48,278::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:08:48,278::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:08:48,278::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:08:48,278::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:08:48,278::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:08:48,278::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:08:48,279::task::993::Storage.TaskManager.Task::(_decref) Task=`9e661a46-3078-400d-b845-2e2397852b0c`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:48,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:48,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112889 s, 90.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:08:50,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:50,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116125 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) (2352) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:50,682::__init__::225::IOProcess::(_processLogs) (2352) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:50,683::__init__::225::IOProcess::(_processLogs) (2352) Queuing response >Thread-21::DEBUG::2014-08-22 10:08:50,684::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:50,684::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:50,691::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:08:50,691::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000187449 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:50,692::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:08:50,692::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:50,692::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:50,692::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:50,693::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:50,693::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:50,693::__init__::225::IOProcess::(_processLogs) (2353) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:50,693::__init__::225::IOProcess::(_processLogs) (2353) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:50,693::__init__::225::IOProcess::(_processLogs) DEBUG|(2353) Building response >Thread-17::DEBUG::2014-08-22 10:08:50,694::__init__::225::IOProcess::(_processLogs) (2353) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:50,694::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:50,694::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:08:52,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:52,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117903 s, 86.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:52,864::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:52,864::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:52,864::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:52,864::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 10:08:52,865::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:08:52,865::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:52,872::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:52,873::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 10:08:52,873::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000219817 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:08:52,874::__init__::225::IOProcess::(_processLogs) (2354) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:52,874::__init__::225::IOProcess::(_processLogs) (2354) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) (2354) Building response >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) (2354) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:52,875::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:52,876::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:52,876::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:52,876::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:52,876::__init__::225::IOProcess::(_processLogs) (2355) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:08:52,876::__init__::225::IOProcess::(_processLogs) (2355) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:08:52,877::__init__::225::IOProcess::(_processLogs) DEBUG|(2355) Building response >Thread-17::DEBUG::2014-08-22 10:08:52,877::__init__::225::IOProcess::(_processLogs) (2355) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:52,877::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:52,877::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:08:52,877::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:52,878::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:08:52,878::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:52,878::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:52,878::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:52,878::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) (2356) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) (2356) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) (2356) Building response >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) (2356) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:08:52,879::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) (2357) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) (2357) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:52,880::__init__::225::IOProcess::(_processLogs) (2357) Building response >Thread-17::DEBUG::2014-08-22 10:08:52,881::__init__::225::IOProcess::(_processLogs) (2357) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:52,881::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:52,881::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:08:52,881::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:08:52,881::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:08:52,881::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) (2358) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) (2358) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:08:52,882::__init__::225::IOProcess::(_processLogs) (2358) Building response >Thread-17::DEBUG::2014-08-22 10:08:52,883::__init__::225::IOProcess::(_processLogs) (2358) Queuing response >Thread-17::DEBUG::2014-08-22 10:08:52,883::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:08:52,883::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:08:54,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:54,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116242 s, 88.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:55,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`77437033-f327-400f-bed6-a0c17fc52699`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:55,874::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:08:55,874::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187449', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219817', 'lastCheck': '3.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:55,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`77437033-f327-400f-bed6-a0c17fc52699`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000187449', 'lastCheck': '5.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000219817', 'lastCheck': '3.0', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:08:55,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`77437033-f327-400f-bed6-a0c17fc52699`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:55,874::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:55,874::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:55,874::task::993::Storage.TaskManager.Task::(_decref) Task=`77437033-f327-400f-bed6-a0c17fc52699`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:56,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:56,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114521 s, 89.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:08:58,327::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:08:58,327::task::595::Storage.TaskManager.Task::(_updateState) Task=`d433d62d-1e7d-4d16-8b23-f16d7979d845`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:08:58,327::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:08:58,328::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:58,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`d433d62d-1e7d-4d16-8b23-f16d7979d845`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:08:58,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`d433d62d-1e7d-4d16-8b23-f16d7979d845`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:08:58,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:08:58,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:08:58,328::task::993::Storage.TaskManager.Task::(_decref) Task=`d433d62d-1e7d-4d16-8b23-f16d7979d845`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:08:58,352::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:08:58,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3c020c7-c91d-41db-83c5-c3c5d48d968a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:08:58,353::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:08:58,353::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`533b4c01-0361-47b0-aa95-27e494c96686`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:08:58,353::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:08:58,353::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:08:58,353::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`533b4c01-0361-47b0-aa95-27e494c96686`::Granted request >Thread-46::DEBUG::2014-08-22 10:08:58,354::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d3c020c7-c91d-41db-83c5-c3c5d48d968a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:08:58,354::task::993::Storage.TaskManager.Task::(_decref) Task=`d3c020c7-c91d-41db-83c5-c3c5d48d968a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:08:58,356::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:58,356::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3c020c7-c91d-41db-83c5-c3c5d48d968a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:08:58,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3c020c7-c91d-41db-83c5-c3c5d48d968a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:08:58,356::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:08:58,356::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:08:58,356::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:08:58,356::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:08:58,356::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:08:58,356::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:08:58,356::task::993::Storage.TaskManager.Task::(_decref) Task=`d3c020c7-c91d-41db-83c5-c3c5d48d968a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:08:58,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:08:58,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011367 s, 90.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:00,695::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:00,695::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:00,695::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:00,696::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:09:00,696::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:00,697::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:00,703::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:09:00,704::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000209283 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:00,704::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:00,705::__init__::225::IOProcess::(_processLogs) (2359) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:00,705::__init__::225::IOProcess::(_processLogs) (2359) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:00,705::__init__::225::IOProcess::(_processLogs) (2359) Building response >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) (2359) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:00,706::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:00,707::__init__::225::IOProcess::(_processLogs) (2360) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:00,707::__init__::225::IOProcess::(_processLogs) (2360) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:00,707::__init__::225::IOProcess::(_processLogs) DEBUG|(2360) Building response >Thread-17::DEBUG::2014-08-22 10:09:00,708::__init__::225::IOProcess::(_processLogs) (2360) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:00,708::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:00,708::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:09:00,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:00,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111602 s, 91.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:02,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:02,884::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:02,886::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:02,887::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:02,888::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:02,889::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:02,890::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:02,891::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:02,919::__init__::225::IOProcess::(_processLogs) DEBUG|(2361) Finding callback 'statvfs'... >Dummy-45::DEBUG::2014-08-22 10:09:02,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241559 s, 42.4 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-08-22 10:09:02,920::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:02,921::__init__::225::IOProcess::(_processLogs) (2361) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:02,928::__init__::225::IOProcess::(_processLogs) (2361) Building response >Thread-17::DEBUG::2014-08-22 10:09:02,930::__init__::225::IOProcess::(_processLogs) (2361) Queuing response >Thread-22::DEBUG::2014-08-22 10:09:02,930::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000184334 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:02,931::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:02,933::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:02,935::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:02,936::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:02,937::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:02,938::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:02,939::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:02,940::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:02,941::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:02,943::__init__::225::IOProcess::(_processLogs) (2362) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:02,947::__init__::225::IOProcess::(_processLogs) DEBUG|(2362) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:02,948::__init__::225::IOProcess::(_processLogs) (2362) Building response >Thread-17::DEBUG::2014-08-22 10:09:02,949::__init__::225::IOProcess::(_processLogs) (2362) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:02,950::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:02,951::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:02,954::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:02,955::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:09:02,956::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:02,957::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:02,958::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:02,959::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:02,960::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:02,961::__init__::225::IOProcess::(_processLogs) (2363) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:02,962::__init__::225::IOProcess::(_processLogs) (2363) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:02,964::__init__::225::IOProcess::(_processLogs) DEBUG|(2363) Building response >Thread-17::DEBUG::2014-08-22 10:09:02,966::__init__::225::IOProcess::(_processLogs) (2363) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:02,966::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:02,966::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:02,967::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:02,967::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:09:02,968::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:02,968::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:02,969::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:02,969::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:02,970::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:02,970::__init__::225::IOProcess::(_processLogs) (2364) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:02,971::__init__::225::IOProcess::(_processLogs) (2364) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:02,971::__init__::225::IOProcess::(_processLogs) (2364) Building response >Thread-17::DEBUG::2014-08-22 10:09:02,972::__init__::225::IOProcess::(_processLogs) (2364) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:02,972::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:02,972::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:02,973::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:02,974::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:09:02,974::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:02,974::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:02,975::__init__::225::IOProcess::(_processLogs) DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:02,976::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:02,977::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:02,977::__init__::225::IOProcess::(_processLogs) (2365) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:02,977::__init__::225::IOProcess::(_processLogs) (2365) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:02,978::__init__::225::IOProcess::(_processLogs) (2365) Building response >Thread-17::DEBUG::2014-08-22 10:09:02,978::__init__::225::IOProcess::(_processLogs) (2365) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:02,978::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:02,979::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:09:04,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:05,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117972 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:07,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:07,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114082 s, 89.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:08,405::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:09:08,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`27737dc8-3633-46fc-aca0-685a4ccbe85e`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:08,406::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:09:08,406::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:08,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`27737dc8-3633-46fc-aca0-685a4ccbe85e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:08,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`27737dc8-3633-46fc-aca0-685a4ccbe85e`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:08,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:08,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:08,406::task::993::Storage.TaskManager.Task::(_decref) Task=`27737dc8-3633-46fc-aca0-685a4ccbe85e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:09:08,440::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:09:08,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`2976e744-8857-43ea-8a2b-aaadf364fa90`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:09:08,440::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:09:08,441::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`33cfd833-3559-46e5-ba16-e288eeea3893`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:09:08,441::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:09:08,441::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:09:08,441::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`33cfd833-3559-46e5-ba16-e288eeea3893`::Granted request >Thread-46::DEBUG::2014-08-22 10:09:08,441::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2976e744-8857-43ea-8a2b-aaadf364fa90`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:09:08,441::task::993::Storage.TaskManager.Task::(_decref) Task=`2976e744-8857-43ea-8a2b-aaadf364fa90`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:09:08,443::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:08,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`2976e744-8857-43ea-8a2b-aaadf364fa90`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:08,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`2976e744-8857-43ea-8a2b-aaadf364fa90`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:09:08,443::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:09:08,444::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:09:08,444::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:09:08,444::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:09:08,444::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:09:08,444::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:09:08,444::task::993::Storage.TaskManager.Task::(_decref) Task=`2976e744-8857-43ea-8a2b-aaadf364fa90`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:09,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:09,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113357 s, 90.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) (2366) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:10,709::__init__::225::IOProcess::(_processLogs) (2366) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:10,710::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2366) Building response >Thread-21::DEBUG::2014-08-22 10:09:10,711::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:10,711::__init__::225::IOProcess::(_processLogs) (2366) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:10,717::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:09:10,718::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000181094 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:10,718::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) (2367) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:10,720::__init__::225::IOProcess::(_processLogs) (2367) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:10,721::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2367) Building response >Thread-17::DEBUG::2014-08-22 10:09:10,722::__init__::225::IOProcess::(_processLogs) (2367) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:10,723::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:10,723::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:09:11,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`079db990-d9f7-4164-b85e-fe013d7c24fa`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:11,115::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:09:11,115::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000181094', 'lastCheck': '0.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000184334', 'lastCheck': '8.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:11,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`079db990-d9f7-4164-b85e-fe013d7c24fa`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000181094', 'lastCheck': '0.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000184334', 'lastCheck': '8.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:11,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`079db990-d9f7-4164-b85e-fe013d7c24fa`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:11,115::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:11,116::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:11,116::task::993::Storage.TaskManager.Task::(_decref) Task=`079db990-d9f7-4164-b85e-fe013d7c24fa`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:11,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:11,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115941 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:12,976::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) (2368) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:12,977::__init__::225::IOProcess::(_processLogs) (2368) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:12,978::__init__::225::IOProcess::(_processLogs) DEBUG|(2368) Building response >Thread-22::DEBUG::2014-08-22 10:09:12,979::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:12,979::__init__::225::IOProcess::(_processLogs) (2368) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:12,985::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:09:12,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000179384 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:12,986::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:12,987::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) (2369) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:12,988::__init__::225::IOProcess::(_processLogs) (2369) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:12,989::__init__::225::IOProcess::(_processLogs) DEBUG|(2369) Building response >Thread-17::DEBUG::2014-08-22 10:09:12,990::__init__::225::IOProcess::(_processLogs) (2369) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:12,990::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:12,990::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:12,990::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:12,990::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:09:12,990::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) (2370) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) (2370) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:12,991::__init__::225::IOProcess::(_processLogs) (2370) Building response >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) (2370) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:12,992::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) (2371) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) (2371) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) (2371) Building response >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) (2371) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:12,993::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:12,994::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:12,994::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:12,994::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:09:12,994::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:12,994::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:12,994::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) (2372) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) (2372) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) (2372) Building response >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) (2372) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:12,995::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:12,996::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:09:13,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:13,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011647 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:15,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:15,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115259 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:17,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:17,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114927 s, 89.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:18,492::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-18::DEBUG::2014-08-22 10:09:18,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`c960c8f7-6165-44f5-a033-68e17c6936dd`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:18,493::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:09:18,493::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:18,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`c960c8f7-6165-44f5-a033-68e17c6936dd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:18,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`c960c8f7-6165-44f5-a033-68e17c6936dd`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:18,494::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:18,494::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:18,494::task::993::Storage.TaskManager.Task::(_decref) Task=`c960c8f7-6165-44f5-a033-68e17c6936dd`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:09:18,569::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [9775f30] >Thread-46::DEBUG::2014-08-22 10:09:18,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`ace0c8c6-d3e1-4bfe-81a6-590eb71c22ff`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:09:18,569::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:09:18,570::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b4abca9a-0851-4ce0-a0af-f1fa59c795cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:09:18,570::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:09:18,570::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:09:18,570::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b4abca9a-0851-4ce0-a0af-f1fa59c795cb`::Granted request >Thread-46::DEBUG::2014-08-22 10:09:18,570::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ace0c8c6-d3e1-4bfe-81a6-590eb71c22ff`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:09:18,570::task::993::Storage.TaskManager.Task::(_decref) Task=`ace0c8c6-d3e1-4bfe-81a6-590eb71c22ff`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:09:18,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:18,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`ace0c8c6-d3e1-4bfe-81a6-590eb71c22ff`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:18,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`ace0c8c6-d3e1-4bfe-81a6-590eb71c22ff`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:09:18,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:09:18,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:09:18,573::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:09:18,573::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:09:18,573::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:09:18,573::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:09:18,573::task::993::Storage.TaskManager.Task::(_decref) Task=`ace0c8c6-d3e1-4bfe-81a6-590eb71c22ff`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:19,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:19,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197484 s, 51.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:20,723::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) (2373) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:20,724::__init__::225::IOProcess::(_processLogs) (2373) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:20,725::__init__::225::IOProcess::(_processLogs) DEBUG|(2373) Building response >Thread-21::DEBUG::2014-08-22 10:09:20,726::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:20,726::__init__::225::IOProcess::(_processLogs) (2373) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:20,732::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:09:20,733::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000201244 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:20,733::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:20,734::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:20,734::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:20,734::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:20,734::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:20,735::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:20,735::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:20,735::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:20,735::__init__::225::IOProcess::(_processLogs) (2374) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:20,735::__init__::225::IOProcess::(_processLogs) (2374) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:20,736::__init__::225::IOProcess::(_processLogs) DEBUG|(2374) Building response >Thread-17::DEBUG::2014-08-22 10:09:20,737::__init__::225::IOProcess::(_processLogs) (2374) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:20,737::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:20,737::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:09:21,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:21,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111971 s, 91.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:22,995::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) (2375) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:22,996::__init__::225::IOProcess::(_processLogs) (2375) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:22,997::__init__::225::IOProcess::(_processLogs) DEBUG|(2375) Building response >Thread-22::DEBUG::2014-08-22 10:09:22,997::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:22,997::__init__::225::IOProcess::(_processLogs) (2375) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:23,004::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:23,005::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:09:23,005::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00022054 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:23,006::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:23,006::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:23,006::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:23,006::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:23,007::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:23,007::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:23,007::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:23,007::__init__::225::IOProcess::(_processLogs) (2376) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:23,007::__init__::225::IOProcess::(_processLogs) (2376) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:23,007::__init__::225::IOProcess::(_processLogs) (2376) Building response >Thread-17::DEBUG::2014-08-22 10:09:23,008::__init__::225::IOProcess::(_processLogs) (2376) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:23,008::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:23,008::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:23,009::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:23,009::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:09:23,009::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:23,009::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) (2377) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) (2377) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) (2377) Building response >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) (2377) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:23,010::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:23,011::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:09:23,011::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:23,011::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:23,011::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:23,011::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:23,011::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) (2378) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) (2378) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) (2378) Building response >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) (2378) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:09:23,012::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:23,013::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:23,013::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) (2379) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) (2379) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) (2379) Building response >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) (2379) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:23,014::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:09:23,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:23,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114514 s, 89.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:25,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:25,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117411 s, 87.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:26,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`43dbc005-180f-46cc-9576-1eccdf3e7d64`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:26,300::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:09:26,300::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000201244', 'lastCheck': '5.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00022054', 'lastCheck': '3.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:26,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`43dbc005-180f-46cc-9576-1eccdf3e7d64`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000201244', 'lastCheck': '5.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00022054', 'lastCheck': '3.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:26,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`43dbc005-180f-46cc-9576-1eccdf3e7d64`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:26,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:26,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:26,300::task::993::Storage.TaskManager.Task::(_decref) Task=`43dbc005-180f-46cc-9576-1eccdf3e7d64`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:27,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:27,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113313 s, 90.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:28,623::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:09:28,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ebc7809-572c-4fe6-a9a8-39ca5bd47e7a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:28,623::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:09:28,623::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:28,623::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ebc7809-572c-4fe6-a9a8-39ca5bd47e7a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:28,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ebc7809-572c-4fe6-a9a8-39ca5bd47e7a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:28,624::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:28,624::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:28,624::task::993::Storage.TaskManager.Task::(_decref) Task=`7ebc7809-572c-4fe6-a9a8-39ca5bd47e7a`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:09:28,689::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:09:28,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae835171-32e3-4635-bfaf-0e445ef36a62`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:09:28,689::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:09:28,690::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`34efdf68-338d-4384-a48e-bd6dbff26e23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:09:28,690::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:09:28,690::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:09:28,690::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`34efdf68-338d-4384-a48e-bd6dbff26e23`::Granted request >Thread-46::DEBUG::2014-08-22 10:09:28,690::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ae835171-32e3-4635-bfaf-0e445ef36a62`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:09:28,690::task::993::Storage.TaskManager.Task::(_decref) Task=`ae835171-32e3-4635-bfaf-0e445ef36a62`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:09:28,692::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:28,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`ae835171-32e3-4635-bfaf-0e445ef36a62`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:28,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae835171-32e3-4635-bfaf-0e445ef36a62`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:09:28,693::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:09:28,693::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:09:28,693::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:09:28,693::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:09:28,693::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:09:28,693::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:09:28,693::task::993::Storage.TaskManager.Task::(_decref) Task=`ae835171-32e3-4635-bfaf-0e445ef36a62`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:29,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:29,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112514 s, 91.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:30,737::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:30,738::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:30,738::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:30,738::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:30,739::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:30,739::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:30,739::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:30,739::__init__::225::IOProcess::(_processLogs) (2380) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:30,739::__init__::225::IOProcess::(_processLogs) (2380) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:30,740::__init__::225::IOProcess::(_processLogs) (2380) Building response >Thread-17::DEBUG::2014-08-22 10:09:30,740::__init__::225::IOProcess::(_processLogs) (2380) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:30,740::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:30,740::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:09:30,740::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 10:09:30,747::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000231896 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:30,747::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:30,748::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:30,748::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) (2381) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) (2381) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) (2381) Building response >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) (2381) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:30,749::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:09:31,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:31,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011746 s, 87.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:33,014::__init__::225::IOProcess::(_processLogs) (2376) Building responseDEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:33,014::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:33,014::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:33,014::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:33,014::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:33,014::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:33,015::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:33,015::__init__::225::IOProcess::(_processLogs) (2382) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:33,015::__init__::225::IOProcess::(_processLogs) (2382) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:33,015::__init__::225::IOProcess::(_processLogs) (2382) Queuing response >Thread-22::DEBUG::2014-08-22 10:09:33,016::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:33,016::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:33,023::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:09:33,024::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000174554 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:33,024::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:33,024::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:33,024::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:33,025::__init__::225::IOProcess::(_processLogs) (2383) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:33,025::__init__::225::IOProcess::(_processLogs) (2383) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:33,025::__init__::225::IOProcess::(_processLogs) (2383) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:33,026::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:33,026::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:33,026::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:09:33,027::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:33,027::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:33,027::__init__::225::IOProcess::(_processLogs) (2384) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:33,027::__init__::225::IOProcess::(_processLogs) (2384) Building response >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) (2384) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:33,028::__init__::225::IOProcess::(_processLogs) (2385) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:33,029::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'...DEBUG|(2385) Building response >Thread-17::DEBUG::2014-08-22 10:09:33,029::__init__::225::IOProcess::(_processLogs) (2385) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:33,029::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:33,029::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:33,029::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'...DEBUG|Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'...DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:33,030::__init__::225::IOProcess::(_processLogs) (2386) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:33,031::__init__::225::IOProcess::(_processLogs) (2386) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:33,031::__init__::225::IOProcess::(_processLogs) (2386) Building response >Thread-17::DEBUG::2014-08-22 10:09:33,031::__init__::225::IOProcess::(_processLogs) (2386) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:33,031::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:33,031::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:09:33,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:33,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116714 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:35,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:35,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114141 s, 89.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:37,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:37,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113645 s, 90.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:38,742::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:09:38,742::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ac23542-866c-472e-8baf-6da7f199ec3b`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:38,742::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:09:38,743::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:38,743::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ac23542-866c-472e-8baf-6da7f199ec3b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:38,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ac23542-866c-472e-8baf-6da7f199ec3b`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:38,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:38,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:38,744::task::993::Storage.TaskManager.Task::(_decref) Task=`5ac23542-866c-472e-8baf-6da7f199ec3b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:09:38,810::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:09:38,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`0880a92f-b36d-4d0b-8cd9-27308f50ccce`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:09:38,810::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:09:38,810::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a6e6d02d-cbaa-4b58-adae-e2d365e8e13f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:09:38,810::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:09:38,810::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:09:38,811::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a6e6d02d-cbaa-4b58-adae-e2d365e8e13f`::Granted request >Thread-46::DEBUG::2014-08-22 10:09:38,811::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0880a92f-b36d-4d0b-8cd9-27308f50ccce`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:09:38,811::task::993::Storage.TaskManager.Task::(_decref) Task=`0880a92f-b36d-4d0b-8cd9-27308f50ccce`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:09:38,813::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:38,813::task::1191::Storage.TaskManager.Task::(prepare) Task=`0880a92f-b36d-4d0b-8cd9-27308f50ccce`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:38,813::task::595::Storage.TaskManager.Task::(_updateState) Task=`0880a92f-b36d-4d0b-8cd9-27308f50ccce`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:09:38,813::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:09:38,813::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:09:38,813::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:09:38,813::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:09:38,813::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:09:38,814::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:09:38,814::task::993::Storage.TaskManager.Task::(_decref) Task=`0880a92f-b36d-4d0b-8cd9-27308f50ccce`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:39,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:39,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117225 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:40,749::__init__::225::IOProcess::(_processLogs) (2385) Finding callback 'access'...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:40,749::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:40,750::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-21::DEBUG::2014-08-22 10:09:40,751::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:40,751::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:40,757::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:40,758::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:09:40,758::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000237336 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:40,759::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:40,759::__init__::225::IOProcess::(_processLogs) (2387) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) (2387) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) (2387) Building response >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) (2387) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:40,760::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:40,761::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:40,761::__init__::225::IOProcess::(_processLogs) (2388) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:40,761::__init__::225::IOProcess::(_processLogs) (2388) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:40,761::__init__::225::IOProcess::(_processLogs) (2388) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:40,762::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:40,762::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:09:41,558::task::595::Storage.TaskManager.Task::(_updateState) Task=`005a07d8-fcd3-40fd-ba97-db788a784f57`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:41,559::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:09:41,559::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000237336', 'lastCheck': '0.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000174554', 'lastCheck': '8.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:41,559::task::1191::Storage.TaskManager.Task::(prepare) Task=`005a07d8-fcd3-40fd-ba97-db788a784f57`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000237336', 'lastCheck': '0.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000174554', 'lastCheck': '8.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:41,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`005a07d8-fcd3-40fd-ba97-db788a784f57`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:41,559::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:41,559::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:41,559::task::993::Storage.TaskManager.Task::(_decref) Task=`005a07d8-fcd3-40fd-ba97-db788a784f57`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:41,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:41,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114674 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:43,031::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:43,032::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:43,032::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:43,032::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:43,032::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:43,032::__init__::225::IOProcess::(_processLogs) (2389) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:43,032::__init__::225::IOProcess::(_processLogs) (2389) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:43,033::__init__::225::IOProcess::(_processLogs) (2389) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:43,033::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:09:43,033::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:43,034::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:09:43,040::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000233824 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:43,041::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:43,041::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:43,041::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:43,041::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:43,041::__init__::225::IOProcess::(_processLogs) (2390) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:43,041::__init__::225::IOProcess::(_processLogs) (2390) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:43,042::__init__::225::IOProcess::(_processLogs) (2390) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:43,042::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:43,042::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:43,043::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:09:43,043::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:43,043::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:43,043::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) (2391) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) (2391) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) (2391) Building response >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) (2391) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:09:43,044::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) (2392) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) (2392) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) (2392) Building response >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) (2392) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:43,045::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:43,046::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:43,046::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:09:43,046::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:43,046::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) (2393) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) (2393) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) (2393) Building response >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) (2393) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:43,047::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:09:43,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:44,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214418 s, 47.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:46,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:46,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113525 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:48,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:48,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134862 s, 75.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:48,862::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-18::DEBUG::2014-08-22 10:09:48,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`351ff45c-6e6c-490c-9977-5a78f56b0598`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:48,863::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:09:48,863::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:48,863::task::1191::Storage.TaskManager.Task::(prepare) Task=`351ff45c-6e6c-490c-9977-5a78f56b0598`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:48,863::task::595::Storage.TaskManager.Task::(_updateState) Task=`351ff45c-6e6c-490c-9977-5a78f56b0598`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:48,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:48,863::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:48,863::task::993::Storage.TaskManager.Task::(_decref) Task=`351ff45c-6e6c-490c-9977-5a78f56b0598`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:09:48,897::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] flowID [3127ed98] >Thread-46::DEBUG::2014-08-22 10:09:48,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`faad1e53-dfe1-4099-8121-5ac005f6f05a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:09:48,897::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:09:48,898::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`10e88936-a4a7-4792-a412-d90d0f6b7d1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:09:48,898::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:09:48,898::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:09:48,898::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`10e88936-a4a7-4792-a412-d90d0f6b7d1e`::Granted request >Thread-46::DEBUG::2014-08-22 10:09:48,898::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`faad1e53-dfe1-4099-8121-5ac005f6f05a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:09:48,898::task::993::Storage.TaskManager.Task::(_decref) Task=`faad1e53-dfe1-4099-8121-5ac005f6f05a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:09:48,900::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:48,900::task::1191::Storage.TaskManager.Task::(prepare) Task=`faad1e53-dfe1-4099-8121-5ac005f6f05a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:48,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`faad1e53-dfe1-4099-8121-5ac005f6f05a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:09:48,901::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:09:48,901::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:09:48,901::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:09:48,901::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:09:48,901::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:09:48,901::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:09:48,901::task::993::Storage.TaskManager.Task::(_decref) Task=`faad1e53-dfe1-4099-8121-5ac005f6f05a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:50,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:50,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135615 s, 75.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:50,763::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:50,763::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:50,763::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:50,763::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:50,764::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:50,764::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:50,765::__init__::225::IOProcess::(_processLogs) (2394) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:50,766::__init__::225::IOProcess::(_processLogs) (2394) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:50,767::__init__::225::IOProcess::(_processLogs) (2394) Queuing response >Thread-21::DEBUG::2014-08-22 10:09:50,768::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:50,768::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:50,774::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:09:50,774::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000169599 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:50,775::__init__::225::IOProcess::(_processLogs) (2395) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:50,776::__init__::225::IOProcess::(_processLogs) (2395) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:50,776::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2395) Building response >Thread-17::DEBUG::2014-08-22 10:09:50,777::__init__::225::IOProcess::(_processLogs) (2395) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:50,777::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:50,777::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:09:52,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:52,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119575 s, 85.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:53,047::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) (2396) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:53,048::__init__::225::IOProcess::(_processLogs) (2396) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:53,049::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2396) Building response >Thread-17::DEBUG::2014-08-22 10:09:53,049::__init__::225::IOProcess::(_processLogs) (2396) Queuing response >Thread-22::DEBUG::2014-08-22 10:09:53,049::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:09:53,050::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:53,056::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:09:53,057::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000225624 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:53,058::__init__::225::IOProcess::(_processLogs) (2397) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:09:53,059::__init__::225::IOProcess::(_processLogs) (2397) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:09:53,059::__init__::225::IOProcess::(_processLogs) (2397) Building response >Thread-17::DEBUG::2014-08-22 10:09:53,059::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2397) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:53,060::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:53,060::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:09:53,060::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:53,060::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:09:53,060::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:53,061::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:53,061::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:53,061::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:53,061::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:53,061::__init__::225::IOProcess::(_processLogs) (2398) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) (2398) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) (2398) Building response >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) (2398) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:09:53,062::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) (2399) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) (2399) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) (2399) Building response >Thread-17::DEBUG::2014-08-22 10:09:53,063::__init__::225::IOProcess::(_processLogs) (2399) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:53,064::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:53,064::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:09:53,064::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:09:53,064::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:09:53,064::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:09:53,064::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) (2400) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) (2400) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) (2400) Building response >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) (2400) Queuing response >Thread-17::DEBUG::2014-08-22 10:09:53,065::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:09:53,066::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:09:54,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:54,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119675 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:09:56,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:56,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114036 s, 89.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:56,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c2e8fc3-2b14-4d82-96c7-b5cb0f969f94`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:56,790::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:09:56,790::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000169599', 'lastCheck': '6.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225624', 'lastCheck': '3.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:56,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c2e8fc3-2b14-4d82-96c7-b5cb0f969f94`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000169599', 'lastCheck': '6.0', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000225624', 'lastCheck': '3.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:09:56,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c2e8fc3-2b14-4d82-96c7-b5cb0f969f94`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:56,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:56,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:56,791::task::993::Storage.TaskManager.Task::(_decref) Task=`0c2e8fc3-2b14-4d82-96c7-b5cb0f969f94`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:09:58,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:09:58,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115923 s, 88.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:09:58,951::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:09:58,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`b212f7c2-e29f-4b82-9ca2-a2af062ee9a2`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:09:58,951::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:09:58,951::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:58,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`b212f7c2-e29f-4b82-9ca2-a2af062ee9a2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:09:58,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`b212f7c2-e29f-4b82-9ca2-a2af062ee9a2`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:09:58,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:09:58,952::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:09:58,952::task::993::Storage.TaskManager.Task::(_decref) Task=`b212f7c2-e29f-4b82-9ca2-a2af062ee9a2`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:09:58,984::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:09:58,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`370196f5-d6d3-44df-85fa-5f5019a7e86b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:09:58,984::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:09:58,985::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e95f654d-13bf-4e14-aec2-5ac4907e29ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:09:58,985::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:09:58,985::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:09:58,985::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`e95f654d-13bf-4e14-aec2-5ac4907e29ed`::Granted request >Thread-46::DEBUG::2014-08-22 10:09:58,985::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`370196f5-d6d3-44df-85fa-5f5019a7e86b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:09:58,986::task::993::Storage.TaskManager.Task::(_decref) Task=`370196f5-d6d3-44df-85fa-5f5019a7e86b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:09:58,987::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:58,988::task::1191::Storage.TaskManager.Task::(prepare) Task=`370196f5-d6d3-44df-85fa-5f5019a7e86b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:09:58,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`370196f5-d6d3-44df-85fa-5f5019a7e86b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:09:58,988::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:09:58,988::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:09:58,988::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:09:58,988::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:09:58,988::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:09:58,988::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:09:58,988::task::993::Storage.TaskManager.Task::(_decref) Task=`370196f5-d6d3-44df-85fa-5f5019a7e86b`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:00,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:00,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114778 s, 89.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) (2401) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:00,778::__init__::225::IOProcess::(_processLogs) (2401) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:00,779::__init__::225::IOProcess::(_processLogs) (2401) Building response >Thread-17::DEBUG::2014-08-22 10:10:00,779::__init__::225::IOProcess::(_processLogs) (2401) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:00,779::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:00,779::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:10:00,780::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-08-22 10:10:00,786::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000251535 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:00,787::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:00,787::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:00,787::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:00,788::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) (2402) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) (2402) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) (2402) Building response >Thread-17::DEBUG::2014-08-22 10:10:00,789::__init__::225::IOProcess::(_processLogs) (2402) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:00,790::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:00,790::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:10:02,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:02,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179215 s, 57.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:03,065::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:03,065::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) (2403) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:03,066::__init__::225::IOProcess::(_processLogs) (2403) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:03,068::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2403) Building response >Thread-22::DEBUG::2014-08-22 10:10:03,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:03,069::__init__::225::IOProcess::(_processLogs) (2403) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:03,075::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:10:03,076::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000218655 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:03,076::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) (2404) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:03,078::__init__::225::IOProcess::(_processLogs) (2404) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:03,079::__init__::225::IOProcess::(_processLogs) DEBUG|(2404) Building response >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) (2404) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:03,080::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) (2405) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) (2405) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) (2405) Building response >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) (2405) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:03,081::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:03,082::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:03,082::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:10:03,082::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:03,082::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) (2406) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) (2406) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) (2406) Building response >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) (2406) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:03,083::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:03,084::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:10:03,084::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:03,084::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:03,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:03,084::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:03,084::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:03,085::__init__::225::IOProcess::(_processLogs) (2407) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:03,085::__init__::225::IOProcess::(_processLogs) (2407) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:03,085::__init__::225::IOProcess::(_processLogs) (2407) Building response >Thread-17::DEBUG::2014-08-22 10:10:03,085::__init__::225::IOProcess::(_processLogs) (2407) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:03,085::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:03,085::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:10:04,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:04,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115613 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:06,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:06,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0142722 s, 71.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:08,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:08,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115314 s, 88.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:09,045::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:10:09,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e98fc21-ecc9-4ade-b1e3-92641913d7f7`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:09,046::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:10:09,046::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:09,046::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e98fc21-ecc9-4ade-b1e3-92641913d7f7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:09,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e98fc21-ecc9-4ade-b1e3-92641913d7f7`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:09,046::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:09,046::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:09,046::task::993::Storage.TaskManager.Task::(_decref) Task=`7e98fc21-ecc9-4ade-b1e3-92641913d7f7`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:10:09,079::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:10:09,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8ff0747-a016-4856-b659-4d703a7d63cc`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:10:09,080::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:10:09,080::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c55e2121-9035-483c-abbe-7c0bb873d654`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:10:09,080::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:10:09,080::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:10:09,080::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`c55e2121-9035-483c-abbe-7c0bb873d654`::Granted request >Thread-46::DEBUG::2014-08-22 10:10:09,081::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e8ff0747-a016-4856-b659-4d703a7d63cc`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:10:09,081::task::993::Storage.TaskManager.Task::(_decref) Task=`e8ff0747-a016-4856-b659-4d703a7d63cc`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:10:09,083::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:09,083::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8ff0747-a016-4856-b659-4d703a7d63cc`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:09,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8ff0747-a016-4856-b659-4d703a7d63cc`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:10:09,083::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:10:09,083::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:10:09,083::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:10:09,083::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:10:09,083::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:10:09,083::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:10:09,084::task::993::Storage.TaskManager.Task::(_decref) Task=`e8ff0747-a016-4856-b659-4d703a7d63cc`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:10,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:10,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113939 s, 89.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:10,790::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:10,791::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:10,792::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:10,792::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:10,793::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:10,793::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:10,796::__init__::225::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:10,797::__init__::225::IOProcess::(_processLogs) (2408) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 10:10:10,797::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:10,803::__init__::225::IOProcess::(_processLogs) (2408) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:10,805::__init__::225::IOProcess::(_processLogs) (2408) Building response >Thread-17::DEBUG::2014-08-22 10:10:10,805::__init__::225::IOProcess::(_processLogs) (2408) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:10,806::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:10:10,807::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00115163 s, 345 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:10,807::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:10,810::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:10,810::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:10,811::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:10,811::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:10,812::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:10,812::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:10,813::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:10,813::__init__::225::IOProcess::(_processLogs) (2409) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:10,814::__init__::225::IOProcess::(_processLogs) (2409) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:10,817::__init__::225::IOProcess::(_processLogs) DEBUG|(2409) Building response >Thread-17::DEBUG::2014-08-22 10:10:10,818::__init__::225::IOProcess::(_processLogs) (2409) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:10,818::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:10,819::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:10:12,009::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0c48e3b-007a-4231-8271-99d606e59b1e`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:12,009::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:10:12,010::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00115163', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218655', 'lastCheck': '8.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:12,010::task::1191::Storage.TaskManager.Task::(prepare) Task=`b0c48e3b-007a-4231-8271-99d606e59b1e`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00115163', 'lastCheck': '1.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000218655', 'lastCheck': '8.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:12,011::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0c48e3b-007a-4231-8271-99d606e59b1e`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:12,011::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:12,011::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:12,011::task::993::Storage.TaskManager.Task::(_decref) Task=`b0c48e3b-007a-4231-8271-99d606e59b1e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:12,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:12,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116938 s, 87.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) (2410) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:13,089::__init__::225::IOProcess::(_processLogs) (2410) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:13,090::__init__::225::IOProcess::(_processLogs) DEBUG|(2410) Building response >Thread-22::DEBUG::2014-08-22 10:10:13,091::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:13,091::__init__::225::IOProcess::(_processLogs) (2410) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:13,097::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:10:13,098::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00020726 s, 2.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:13,098::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) (2411) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:13,100::__init__::225::IOProcess::(_processLogs) (2411) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:13,101::__init__::225::IOProcess::(_processLogs) (2411) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:13,102::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:13,102::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:13,102::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:10:13,102::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) (2412) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) (2412) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) (2412) Building response >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) (2412) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:13,103::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:13,104::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:10:13,104::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:13,104::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:13,104::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:13,104::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) (2413) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) (2413) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) (2413) Building response >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) (2413) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:10:13,105::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:13,106::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:13,106::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:13,106::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:13,106::__init__::225::IOProcess::(_processLogs) (2414) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:13,107::__init__::225::IOProcess::(_processLogs) (2414) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:13,107::__init__::225::IOProcess::(_processLogs) (2414) Building response >Thread-17::DEBUG::2014-08-22 10:10:13,107::__init__::225::IOProcess::(_processLogs) (2414) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:13,107::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:13,107::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:10:14,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:14,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121723 s, 84.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:16,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:16,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117536 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:18,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:19,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114574 s, 89.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:19,132::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:10:19,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e451f5e-9947-48bc-8243-5b6a0e2ded79`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:19,133::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:10:19,133::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:19,133::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e451f5e-9947-48bc-8243-5b6a0e2ded79`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:19,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e451f5e-9947-48bc-8243-5b6a0e2ded79`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:19,133::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:19,133::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:19,133::task::993::Storage.TaskManager.Task::(_decref) Task=`0e451f5e-9947-48bc-8243-5b6a0e2ded79`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:10:19,158::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:10:19,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`1dabd510-d800-4e28-84e4-b01ce4e25ce6`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:10:19,159::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:10:19,159::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`01359834-ce72-46e5-a889-4b3dd2298538`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:10:19,159::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:10:19,159::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:10:19,159::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`01359834-ce72-46e5-a889-4b3dd2298538`::Granted request >Thread-46::DEBUG::2014-08-22 10:10:19,160::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1dabd510-d800-4e28-84e4-b01ce4e25ce6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:10:19,160::task::993::Storage.TaskManager.Task::(_decref) Task=`1dabd510-d800-4e28-84e4-b01ce4e25ce6`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:10:19,162::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:19,162::task::1191::Storage.TaskManager.Task::(prepare) Task=`1dabd510-d800-4e28-84e4-b01ce4e25ce6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:19,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`1dabd510-d800-4e28-84e4-b01ce4e25ce6`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:10:19,163::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:10:19,163::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:10:19,163::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:10:19,163::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:10:19,163::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:10:19,163::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:10:19,163::task::993::Storage.TaskManager.Task::(_decref) Task=`1dabd510-d800-4e28-84e4-b01ce4e25ce6`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:10:20,818::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:20,818::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:20,818::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:20,819::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:20,819::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:20,819::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:20,819::__init__::225::IOProcess::(_processLogs) (2415) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:20,819::__init__::225::IOProcess::(_processLogs) (2415) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:20,819::__init__::225::IOProcess::(_processLogs) DEBUG|(2415) Building response >Thread-17::DEBUG::2014-08-22 10:10:20,820::__init__::225::IOProcess::(_processLogs) (2415) Queuing response >Thread-21::DEBUG::2014-08-22 10:10:20,820::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:20,821::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:20,827::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:10:20,828::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000198275 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:20,829::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:20,830::__init__::225::IOProcess::(_processLogs) (2416) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:20,830::__init__::225::IOProcess::(_processLogs) (2416) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:20,830::__init__::225::IOProcess::(_processLogs) (2416) Building response >Thread-17::DEBUG::2014-08-22 10:10:20,830::__init__::225::IOProcess::(_processLogs) (2416) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:20,830::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:20,830::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:10:21,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:21,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118051 s, 86.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:23,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:23,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112811 s, 90.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:23,120::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:23,121::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:23,121::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:23,122::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:23,122::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:23,123::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:23,124::__init__::225::IOProcess::(_processLogs) (2417) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:23,124::__init__::225::IOProcess::(_processLogs) (2417) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:23,127::__init__::225::IOProcess::(_processLogs) (2417) Queuing response >Thread-22::DEBUG::2014-08-22 10:10:23,128::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:23,129::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:23,136::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:10:23,137::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000185042 s, 3.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:23,138::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:23,139::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:23,139::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:23,140::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:23,142::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:23,143::__init__::225::IOProcess::(_processLogs) (2418) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:23,143::__init__::225::IOProcess::(_processLogs) (2418) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:23,144::__init__::225::IOProcess::(_processLogs) (2418) Building response >Thread-17::DEBUG::2014-08-22 10:10:23,144::__init__::225::IOProcess::(_processLogs) (2418) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:23,144::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:23,145::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:23,146::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:10:23,146::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:23,146::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:23,147::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:23,147::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:23,148::__init__::225::IOProcess::(_processLogs) (2419) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:23,149::__init__::225::IOProcess::(_processLogs) (2419) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:23,149::__init__::225::IOProcess::(_processLogs) (2419) Building response >Thread-17::DEBUG::2014-08-22 10:10:23,150::__init__::225::IOProcess::(_processLogs) (2419) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:23,150::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:23,150::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:23,151::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:10:23,152::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:23,152::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:23,152::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:23,153::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:23,154::__init__::225::IOProcess::(_processLogs) (2420) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:23,154::__init__::225::IOProcess::(_processLogs) (2420) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:23,155::__init__::225::IOProcess::(_processLogs) (2420) Building response >Thread-17::DEBUG::2014-08-22 10:10:23,155::__init__::225::IOProcess::(_processLogs) (2420) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:23,156::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:23,156::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:23,157::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:10:23,157::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:23,158::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:23,158::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:23,159::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:23,160::__init__::225::IOProcess::(_processLogs) (2421) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:23,161::__init__::225::IOProcess::(_processLogs) (2421) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:23,161::__init__::225::IOProcess::(_processLogs) (2421) Building response >Thread-17::DEBUG::2014-08-22 10:10:23,161::__init__::225::IOProcess::(_processLogs) (2421) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:23,162::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:23,162::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:10:25,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:25,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116517 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:27,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-18::DEBUG::2014-08-22 10:10:27,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`81b0753a-e484-41f6-8e27-9e5e9ee64f03`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:27,227::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:10:27,227::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000198275', 'lastCheck': '6.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185042', 'lastCheck': '4.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:27,227::task::1191::Storage.TaskManager.Task::(prepare) Task=`81b0753a-e484-41f6-8e27-9e5e9ee64f03`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000198275', 'lastCheck': '6.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000185042', 'lastCheck': '4.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:27,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`81b0753a-e484-41f6-8e27-9e5e9ee64f03`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:27,227::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:27,227::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:27,228::task::993::Storage.TaskManager.Task::(_decref) Task=`81b0753a-e484-41f6-8e27-9e5e9ee64f03`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:27,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112811 s, 90.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:29,241::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:10:29,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`285037f3-5ffc-4855-b71c-a636e9d35376`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:29,242::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:10:29,242::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:29,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`285037f3-5ffc-4855-b71c-a636e9d35376`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:29,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`285037f3-5ffc-4855-b71c-a636e9d35376`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:29,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:29,243::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:29,243::task::993::Storage.TaskManager.Task::(_decref) Task=`285037f3-5ffc-4855-b71c-a636e9d35376`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:29,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-46::DEBUG::2014-08-22 10:10:29,279::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:10:29,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`433e5f31-6f2b-45c2-9e97-f1114e29ca24`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:10:29,280::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:10:29,280::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b036de33-a830-4521-b9b3-b1010e840d83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:10:29,280::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:10:29,281::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:10:29,281::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b036de33-a830-4521-b9b3-b1010e840d83`::Granted request >Thread-46::DEBUG::2014-08-22 10:10:29,281::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`433e5f31-6f2b-45c2-9e97-f1114e29ca24`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:10:29,281::task::993::Storage.TaskManager.Task::(_decref) Task=`433e5f31-6f2b-45c2-9e97-f1114e29ca24`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:10:29,283::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:29,283::task::1191::Storage.TaskManager.Task::(prepare) Task=`433e5f31-6f2b-45c2-9e97-f1114e29ca24`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:29,283::task::595::Storage.TaskManager.Task::(_updateState) Task=`433e5f31-6f2b-45c2-9e97-f1114e29ca24`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:10:29,283::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:10:29,284::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:10:29,284::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:10:29,284::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:10:29,284::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:10:29,284::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:10:29,284::task::993::Storage.TaskManager.Task::(_decref) Task=`433e5f31-6f2b-45c2-9e97-f1114e29ca24`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:29,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0151223 s, 67.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:30,831::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:30,831::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:30,832::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 10:10:30,833::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:30,833::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:30,839::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-08-22 10:10:30,840::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000232226 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:30,840::__init__::225::IOProcess::(_processLogs) (2422) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:30,841::__init__::225::IOProcess::(_processLogs) (2422) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) (2422) Building response >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) (2422) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:30,842::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:30,843::__init__::225::IOProcess::(_processLogs) (2423) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:30,843::__init__::225::IOProcess::(_processLogs) (2423) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:30,843::__init__::225::IOProcess::(_processLogs) (2423) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:30,844::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:30,844::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:10:31,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:31,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116561 s, 87.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:33,160::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) (2424) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:33,161::__init__::225::IOProcess::(_processLogs) (2424) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:33,162::__init__::225::IOProcess::(_processLogs) DEBUG|(2424) Building response >Thread-22::DEBUG::2014-08-22 10:10:33,162::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:33,163::__init__::225::IOProcess::(_processLogs) (2424) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:33,169::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:10:33,170::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.0002547 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:33,170::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:33,171::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:33,171::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) (2425) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) (2425) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) (2425) Building response >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) (2425) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:33,172::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:33,173::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:33,174::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:33,174::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:10:33,174::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:33,174::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:33,174::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) (2426) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) (2426) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) (2426) Building response >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) (2426) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:10:33,175::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:33,176::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:33,176::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:33,176::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:33,176::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:33,176::__init__::225::IOProcess::(_processLogs) (2427) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:33,176::__init__::225::IOProcess::(_processLogs) (2427) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) (2427) Building response >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) (2427) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:33,177::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) (2428) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) (2428) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) (2428) Building response >Thread-17::DEBUG::2014-08-22 10:10:33,178::__init__::225::IOProcess::(_processLogs) (2428) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:33,179::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:33,179::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:10:33,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:33,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117159 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:35,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:35,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114911 s, 89.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:37,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:37,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116344 s, 88.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:39,340::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:10:39,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`9816c38b-fdf0-48f1-b4d2-b37bac4dedc8`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:39,341::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:10:39,341::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:39,341::task::1191::Storage.TaskManager.Task::(prepare) Task=`9816c38b-fdf0-48f1-b4d2-b37bac4dedc8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:39,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`9816c38b-fdf0-48f1-b4d2-b37bac4dedc8`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:39,341::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:39,341::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:39,341::task::993::Storage.TaskManager.Task::(_decref) Task=`9816c38b-fdf0-48f1-b4d2-b37bac4dedc8`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:10:39,431::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:10:39,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`f53974ed-7ba3-4dc1-a591-e8182a5be555`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:10:39,431::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:10:39,431::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9b8444da-d475-48c8-9286-1eca79c909cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:10:39,432::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:10:39,432::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:10:39,432::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`9b8444da-d475-48c8-9286-1eca79c909cf`::Granted request >Thread-46::DEBUG::2014-08-22 10:10:39,432::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f53974ed-7ba3-4dc1-a591-e8182a5be555`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:10:39,432::task::993::Storage.TaskManager.Task::(_decref) Task=`f53974ed-7ba3-4dc1-a591-e8182a5be555`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:10:39,434::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:39,434::task::1191::Storage.TaskManager.Task::(prepare) Task=`f53974ed-7ba3-4dc1-a591-e8182a5be555`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:39,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`f53974ed-7ba3-4dc1-a591-e8182a5be555`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:10:39,434::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:10:39,434::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:10:39,435::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:10:39,435::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:10:39,435::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:10:39,435::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:10:39,435::task::993::Storage.TaskManager.Task::(_decref) Task=`f53974ed-7ba3-4dc1-a591-e8182a5be555`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:39,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:39,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011346 s, 90.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:40,845::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:40,845::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:40,846::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:40,847::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:10:40,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:40,848::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:40,854::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:10:40,855::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000158276 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:40,855::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:40,856::__init__::225::IOProcess::(_processLogs) (2429) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) (2429) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) (2429) Building response >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) (2429) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:40,857::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:40,858::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:40,858::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:40,858::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:40,858::__init__::225::IOProcess::(_processLogs) (2430) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:40,858::__init__::225::IOProcess::(_processLogs) (2430) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:40,858::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2430) Building response >Thread-17::DEBUG::2014-08-22 10:10:40,859::__init__::225::IOProcess::(_processLogs) (2430) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:40,859::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:40,859::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:10:41,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:41,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116325 s, 88.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:42,535::task::595::Storage.TaskManager.Task::(_updateState) Task=`84d588a6-a5b4-44c9-aac6-efc6fad2ef85`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:42,535::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:10:42,535::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000158276', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002547', 'lastCheck': '9.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:42,536::task::1191::Storage.TaskManager.Task::(prepare) Task=`84d588a6-a5b4-44c9-aac6-efc6fad2ef85`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000158276', 'lastCheck': '1.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0002547', 'lastCheck': '9.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:42,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`84d588a6-a5b4-44c9-aac6-efc6fad2ef85`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:42,536::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:42,536::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:42,536::task::993::Storage.TaskManager.Task::(_decref) Task=`84d588a6-a5b4-44c9-aac6-efc6fad2ef85`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:10:43,179::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:43,179::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:43,179::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:43,180::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-22::DEBUG::2014-08-22 10:10:43,180::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:43,180::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:43,187::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:10:43,187::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000166717 s, 3.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:43,188::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) (2431) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) (2431) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) (2431) Building response >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) (2431) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:43,189::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) (2432) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) (2432) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) (2432) Building response >Thread-17::DEBUG::2014-08-22 10:10:43,190::__init__::225::IOProcess::(_processLogs) (2432) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:43,191::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:10:43,191::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:43,192::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:43,192::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:10:43,192::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:43,192::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:43,192::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:43,192::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) (2433) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) (2433) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) (2433) Building response >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) (2433) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:10:43,193::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:43,194::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:43,194::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:43,194::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:43,194::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:43,194::__init__::225::IOProcess::(_processLogs) (2434) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:43,194::__init__::225::IOProcess::(_processLogs) (2434) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) (2434) Building response >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) (2434) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:10:43,195::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:43,196::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:43,196::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:43,196::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:43,196::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:43,196::__init__::225::IOProcess::(_processLogs) (2435) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:43,197::__init__::225::IOProcess::(_processLogs) (2435) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:43,197::__init__::225::IOProcess::(_processLogs) (2435) Building response >Thread-17::DEBUG::2014-08-22 10:10:43,197::__init__::225::IOProcess::(_processLogs) (2435) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:43,197::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:43,197::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:10:43,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:43,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114324 s, 89.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:45,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:45,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115728 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:47,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:47,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114594 s, 89.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:49,561::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:10:49,561::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7a18d66-aff9-4b12-a3f4-f0b843155678`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:49,561::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:10:49,561::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:49,561::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7a18d66-aff9-4b12-a3f4-f0b843155678`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:49,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7a18d66-aff9-4b12-a3f4-f0b843155678`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:49,562::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:49,562::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:49,562::task::993::Storage.TaskManager.Task::(_decref) Task=`c7a18d66-aff9-4b12-a3f4-f0b843155678`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:10:49,587::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:10:49,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f067480-e217-4575-a21b-78bc3ae9dc4e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:10:49,587::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:10:49,587::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7b4a2f72-f0aa-4751-877f-6bca0b87781c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:10:49,587::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:10:49,587::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:10:49,587::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`7b4a2f72-f0aa-4751-877f-6bca0b87781c`::Granted request >Thread-46::DEBUG::2014-08-22 10:10:49,588::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8f067480-e217-4575-a21b-78bc3ae9dc4e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:10:49,588::task::993::Storage.TaskManager.Task::(_decref) Task=`8f067480-e217-4575-a21b-78bc3ae9dc4e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:10:49,590::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:49,590::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f067480-e217-4575-a21b-78bc3ae9dc4e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:49,590::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f067480-e217-4575-a21b-78bc3ae9dc4e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:10:49,590::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:10:49,590::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:10:49,590::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:10:49,590::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:10:49,590::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:10:49,590::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:10:49,590::task::993::Storage.TaskManager.Task::(_decref) Task=`8f067480-e217-4575-a21b-78bc3ae9dc4e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:49,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:49,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011977 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:50,860::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:50,860::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:50,860::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:50,860::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:50,860::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:50,860::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:50,861::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:50,861::__init__::225::IOProcess::(_processLogs) (2436) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:50,861::__init__::225::IOProcess::(_processLogs) (2436) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:50,861::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2436) Building response >Thread-21::DEBUG::2014-08-22 10:10:50,862::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:50,862::__init__::225::IOProcess::(_processLogs) (2436) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:50,869::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:10:50,869::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000205498 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:50,870::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:50,871::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:50,872::__init__::225::IOProcess::(_processLogs) (2437) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:50,872::__init__::225::IOProcess::(_processLogs) (2437) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:50,872::__init__::225::IOProcess::(_processLogs) (2437) Building response >Thread-17::DEBUG::2014-08-22 10:10:50,872::__init__::225::IOProcess::(_processLogs) (2437) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:50,872::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:10:50,873::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:10:51,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:51,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119961 s, 85.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:53,197::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:53,197::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:53,197::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:53,198::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 10:10:53,199::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:10:53,199::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:53,205::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:10:53,206::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00017724 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:10:53,206::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:53,207::__init__::225::IOProcess::(_processLogs) (2438) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:53,207::__init__::225::IOProcess::(_processLogs) (2438) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) (2438) Building response >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) (2438) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:53,208::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:53,209::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:53,209::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:53,209::__init__::225::IOProcess::(_processLogs) (2439) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:10:53,209::__init__::225::IOProcess::(_processLogs) (2439) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:10:53,209::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2439) Building response >Thread-17::DEBUG::2014-08-22 10:10:53,210::__init__::225::IOProcess::(_processLogs) (2439) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:53,210::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:53,210::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:10:53,210::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:53,211::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:10:53,211::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:53,211::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:53,211::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:53,211::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:53,211::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) (2440) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) (2440) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) (2440) Building response >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) (2440) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:10:53,212::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) (2441) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) (2441) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:53,213::__init__::225::IOProcess::(_processLogs) (2441) Building response >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) (2441) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:10:53,214::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) (2442) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) (2442) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) (2442) Building response >Thread-17::DEBUG::2014-08-22 10:10:53,215::__init__::225::IOProcess::(_processLogs) (2442) Queuing response >Thread-17::DEBUG::2014-08-22 10:10:53,216::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:10:53,216::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:10:53,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:53,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0126891 s, 80.7 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:10:55,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:55,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178943 s, 57.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:57,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`0704957e-af26-44ce-959f-77a872c3b9af`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:57,786::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:10:57,787::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205498', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00017724', 'lastCheck': '4.6', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:57,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`0704957e-af26-44ce-959f-77a872c3b9af`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000205498', 'lastCheck': '6.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00017724', 'lastCheck': '4.6', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:10:57,787::task::595::Storage.TaskManager.Task::(_updateState) Task=`0704957e-af26-44ce-959f-77a872c3b9af`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:57,787::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:57,787::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:57,787::task::993::Storage.TaskManager.Task::(_decref) Task=`0704957e-af26-44ce-959f-77a872c3b9af`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:10:57,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:10:58,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011539 s, 88.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:10:59,690::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:10:59,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`80e3c88f-77ad-4ae5-bfec-3d46b9e8738b`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:10:59,690::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:10:59,691::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:59,691::task::1191::Storage.TaskManager.Task::(prepare) Task=`80e3c88f-77ad-4ae5-bfec-3d46b9e8738b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:10:59,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`80e3c88f-77ad-4ae5-bfec-3d46b9e8738b`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:10:59,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:10:59,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:10:59,691::task::993::Storage.TaskManager.Task::(_decref) Task=`80e3c88f-77ad-4ae5-bfec-3d46b9e8738b`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:10:59,738::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:10:59,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`42107d9e-43d9-4278-9232-e921e026246f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:10:59,738::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:10:59,739::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a874eaad-ef9d-4d1e-b83d-4df5a651ce58`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:10:59,739::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:10:59,739::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:10:59,739::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a874eaad-ef9d-4d1e-b83d-4df5a651ce58`::Granted request >Thread-46::DEBUG::2014-08-22 10:10:59,739::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`42107d9e-43d9-4278-9232-e921e026246f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:10:59,739::task::993::Storage.TaskManager.Task::(_decref) Task=`42107d9e-43d9-4278-9232-e921e026246f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:10:59,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:59,741::task::1191::Storage.TaskManager.Task::(prepare) Task=`42107d9e-43d9-4278-9232-e921e026246f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:10:59,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`42107d9e-43d9-4278-9232-e921e026246f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:10:59,742::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:10:59,742::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:10:59,742::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:10:59,742::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:10:59,742::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:10:59,742::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:10:59,742::task::993::Storage.TaskManager.Task::(_decref) Task=`42107d9e-43d9-4278-9232-e921e026246f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:00,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:00,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014357 s, 71.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:00,874::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:00,874::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:00,874::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:00,875::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:11:00,876::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:00,876::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:00,882::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:11:00,883::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000181626 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:00,883::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) (2443) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) (2443) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) (2443) Building response >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) (2443) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:00,885::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:00,886::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:00,887::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:00,887::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:00,887::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:00,887::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:00,887::__init__::225::IOProcess::(_processLogs) (2444) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:00,887::__init__::225::IOProcess::(_processLogs) (2444) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:00,888::__init__::225::IOProcess::(_processLogs) (2444) Building response >Thread-17::DEBUG::2014-08-22 10:11:00,888::__init__::225::IOProcess::(_processLogs) (2444) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:00,888::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:00,888::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:11:02,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:02,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178634 s, 57.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:03,215::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:03,216::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:03,216::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:03,216::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 10:11:03,217::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:03,217::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:03,223::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:11:03,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000179157 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:03,224::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:03,225::__init__::225::IOProcess::(_processLogs) (2445) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:03,225::__init__::225::IOProcess::(_processLogs) (2445) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) (2445) Building response >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) (2445) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:03,226::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:03,227::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:03,227::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:03,227::__init__::225::IOProcess::(_processLogs) (2446) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:03,227::__init__::225::IOProcess::(_processLogs) (2446) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:03,227::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2446) Building response >Thread-17::DEBUG::2014-08-22 10:11:03,228::__init__::225::IOProcess::(_processLogs) (2446) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:03,228::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:03,228::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:03,228::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:03,229::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:11:03,229::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:03,229::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:03,229::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:03,229::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:03,229::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) (2447) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) (2447) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) (2447) Building response >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) (2447) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:11:03,230::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) (2448) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) (2448) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:03,231::__init__::225::IOProcess::(_processLogs) (2448) Building response >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) (2448) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:03,232::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) (2449) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) (2449) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) (2449) Building response >Thread-17::DEBUG::2014-08-22 10:11:03,233::__init__::225::IOProcess::(_processLogs) (2449) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:03,234::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:03,234::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:11:04,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:04,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116896 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:06,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:06,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136816 s, 74.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:08,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:08,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113993 s, 89.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:09,797::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:11:09,798::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6fbff04-3602-4ee9-b0c0-e54e1b651759`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:09,798::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:11:09,798::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:09,798::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6fbff04-3602-4ee9-b0c0-e54e1b651759`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:09,798::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6fbff04-3602-4ee9-b0c0-e54e1b651759`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:09,798::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:09,798::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:09,798::task::993::Storage.TaskManager.Task::(_decref) Task=`a6fbff04-3602-4ee9-b0c0-e54e1b651759`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:11:09,855::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:11:09,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`876f6db3-bdb5-4cbb-b5cd-cff10ae2f29a`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:11:09,856::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:11:09,856::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`21379852-c385-4902-b15e-f03db4043a0b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:11:09,856::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:11:09,856::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:11:09,856::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`21379852-c385-4902-b15e-f03db4043a0b`::Granted request >Thread-46::DEBUG::2014-08-22 10:11:09,856::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`876f6db3-bdb5-4cbb-b5cd-cff10ae2f29a`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:11:09,856::task::993::Storage.TaskManager.Task::(_decref) Task=`876f6db3-bdb5-4cbb-b5cd-cff10ae2f29a`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:11:09,858::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:09,859::task::1191::Storage.TaskManager.Task::(prepare) Task=`876f6db3-bdb5-4cbb-b5cd-cff10ae2f29a`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:09,859::task::595::Storage.TaskManager.Task::(_updateState) Task=`876f6db3-bdb5-4cbb-b5cd-cff10ae2f29a`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:11:09,859::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:11:09,859::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:11:09,859::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:11:09,859::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:11:09,859::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:11:09,859::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:11:09,859::task::993::Storage.TaskManager.Task::(_decref) Task=`876f6db3-bdb5-4cbb-b5cd-cff10ae2f29a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:10,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:10,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115346 s, 88.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:10,888::__init__::225::IOProcess::(_processLogs) (2450) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:10,889::__init__::225::IOProcess::(_processLogs) (2450) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:10,889::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2450) Building response >Thread-21::DEBUG::2014-08-22 10:11:10,890::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:10,890::__init__::225::IOProcess::(_processLogs) (2450) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:10,897::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::DEBUG::2014-08-22 10:11:10,898::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000210304 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:10,898::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:10,899::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:10,899::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:10,899::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:10,899::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:10,899::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:10,900::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:10,900::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:10,900::__init__::225::IOProcess::(_processLogs) (2451) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:10,900::__init__::225::IOProcess::(_processLogs) (2451) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:10,900::__init__::225::IOProcess::(_processLogs) (2451) Building response >Thread-17::DEBUG::2014-08-22 10:11:10,900::__init__::225::IOProcess::(_processLogs) (2451) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:10,901::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:11:10,901::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:11:12,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:12,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116311 s, 88.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:12,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`37430472-f269-40db-bd90-1ae40f2856d3`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:12,998::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:11:12,998::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210304', 'lastCheck': '2.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000179157', 'lastCheck': '9.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:12,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`37430472-f269-40db-bd90-1ae40f2856d3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000210304', 'lastCheck': '2.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000179157', 'lastCheck': '9.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:12,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`37430472-f269-40db-bd90-1ae40f2856d3`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:12,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:12,998::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:12,998::task::993::Storage.TaskManager.Task::(_decref) Task=`37430472-f269-40db-bd90-1ae40f2856d3`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:11:13,233::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) (2452) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:13,234::__init__::225::IOProcess::(_processLogs) (2452) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:13,235::__init__::225::IOProcess::(_processLogs) (2452) Queuing response >Thread-22::DEBUG::2014-08-22 10:11:13,235::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:13,235::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:13,242::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:11:13,243::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000160273 s, 3.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:13,243::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:13,243::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:13,243::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:13,243::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:13,244::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:13,244::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:13,244::__init__::225::IOProcess::(_processLogs) (2453) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:13,244::__init__::225::IOProcess::(_processLogs) (2453) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:13,244::__init__::225::IOProcess::(_processLogs) DEBUG|(2453) Building response >Thread-17::DEBUG::2014-08-22 10:11:13,245::__init__::225::IOProcess::(_processLogs) (2453) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:13,245::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:13,245::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:13,245::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:13,246::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:11:13,246::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:13,246::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:13,246::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:13,246::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) (2454) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) (2454) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) (2454) Building response >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) (2454) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:11:13,247::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:13,248::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:13,248::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:13,248::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:13,248::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:13,248::__init__::225::IOProcess::(_processLogs) (2455) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) (2455) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) (2455) Building response >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) (2455) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:11:13,249::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:13,250::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:13,250::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) (2456) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) (2456) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) (2456) Building response >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) (2456) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:13,251::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:11:14,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:14,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011568 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:16,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:16,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114637 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:18,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:18,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011499 s, 89.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:19,911::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:11:19,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a970b84-d1f0-4364-8c17-a9853b0a4624`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:19,911::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:11:19,912::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:19,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a970b84-d1f0-4364-8c17-a9853b0a4624`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:19,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a970b84-d1f0-4364-8c17-a9853b0a4624`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:19,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:19,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:19,912::task::993::Storage.TaskManager.Task::(_decref) Task=`7a970b84-d1f0-4364-8c17-a9853b0a4624`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:11:19,948::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:11:19,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6c06d26-993e-4a22-8a76-0e7fbc9afe23`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:11:19,949::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:11:19,949::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`83bed31a-1eea-49fb-81c1-16e8a2db4261`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:11:19,949::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:11:19,949::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:11:19,949::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`83bed31a-1eea-49fb-81c1-16e8a2db4261`::Granted request >Thread-46::DEBUG::2014-08-22 10:11:19,950::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e6c06d26-993e-4a22-8a76-0e7fbc9afe23`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:11:19,950::task::993::Storage.TaskManager.Task::(_decref) Task=`e6c06d26-993e-4a22-8a76-0e7fbc9afe23`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:11:19,952::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:19,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6c06d26-993e-4a22-8a76-0e7fbc9afe23`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:19,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6c06d26-993e-4a22-8a76-0e7fbc9afe23`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:11:19,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:11:19,952::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:11:19,952::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:11:19,952::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:11:19,952::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:11:19,952::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:11:19,952::task::993::Storage.TaskManager.Task::(_decref) Task=`e6c06d26-993e-4a22-8a76-0e7fbc9afe23`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:20,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:20,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113067 s, 90.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:20,902::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:20,902::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:20,902::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:20,903::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:11:20,904::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:20,904::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:20,911::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:11:20,912::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000266736 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:20,912::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) (2457) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) (2457) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) (2457) Building response >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) (2457) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:20,913::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) (2458) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) (2458) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) (2458) Building response >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) (2458) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:20,914::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:20,915::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:11:22,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:22,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118739 s, 86.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:23,251::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:23,251::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:23,251::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:23,251::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:23,252::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:23,252::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:23,252::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:23,252::__init__::225::IOProcess::(_processLogs) (2459) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:23,252::__init__::225::IOProcess::(_processLogs) (2459) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:23,252::__init__::225::IOProcess::(_processLogs) DEBUG|(2459) Building response >Thread-22::DEBUG::2014-08-22 10:11:23,253::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:23,253::__init__::225::IOProcess::(_processLogs) (2459) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:23,260::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:23,260::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:11:23,261::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000223524 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:23,262::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:23,262::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:23,262::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:23,262::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:23,262::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:23,262::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:23,263::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:23,263::__init__::225::IOProcess::(_processLogs) (2460) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:23,263::__init__::225::IOProcess::(_processLogs) (2460) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:23,263::__init__::225::IOProcess::(_processLogs) DEBUG|(2460) Building response >Thread-17::DEBUG::2014-08-22 10:11:23,264::__init__::225::IOProcess::(_processLogs) (2460) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:23,264::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:23,264::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:23,264::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:23,265::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:11:23,265::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:23,265::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:23,265::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:23,265::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:23,265::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) (2461) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) (2461) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) (2461) Building response >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) (2461) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:11:23,266::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) (2462) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) (2462) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:23,267::__init__::225::IOProcess::(_processLogs) (2462) Building response >Thread-17::DEBUG::2014-08-22 10:11:23,268::__init__::225::IOProcess::(_processLogs) (2462) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:23,268::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:23,268::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:23,268::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:23,268::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:11:23,268::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) (2463) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) (2463) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) (2463) Building response >Thread-17::DEBUG::2014-08-22 10:11:23,269::__init__::225::IOProcess::(_processLogs) (2463) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:23,270::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:23,270::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:11:24,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:24,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115697 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:26,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:26,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119565 s, 85.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:28,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`34f3e958-cd93-45d1-81e0-f163ae93b907`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:28,179::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:11:28,180::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000266736', 'lastCheck': '7.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223524', 'lastCheck': '4.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:28,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`34f3e958-cd93-45d1-81e0-f163ae93b907`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000266736', 'lastCheck': '7.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000223524', 'lastCheck': '4.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:28,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`34f3e958-cd93-45d1-81e0-f163ae93b907`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:28,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:28,180::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:28,180::task::993::Storage.TaskManager.Task::(_decref) Task=`34f3e958-cd93-45d1-81e0-f163ae93b907`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:28,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:28,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117552 s, 87.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:30,024::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:11:30,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bfa496a-f964-467d-aacd-d186a54e5985`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:30,028::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:11:30,028::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:30,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bfa496a-f964-467d-aacd-d186a54e5985`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:30,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bfa496a-f964-467d-aacd-d186a54e5985`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:30,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:30,028::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:30,028::task::993::Storage.TaskManager.Task::(_decref) Task=`9bfa496a-f964-467d-aacd-d186a54e5985`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:11:30,087::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:11:30,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab30c10-9cf3-494b-9fff-f6f4fb421c25`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:11:30,088::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:11:30,088::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`85d8994b-e8e4-440a-9dbe-ac6f621c9278`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:11:30,089::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:11:30,089::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:11:30,089::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`85d8994b-e8e4-440a-9dbe-ac6f621c9278`::Granted request >Thread-46::DEBUG::2014-08-22 10:11:30,091::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bab30c10-9cf3-494b-9fff-f6f4fb421c25`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:11:30,091::task::993::Storage.TaskManager.Task::(_decref) Task=`bab30c10-9cf3-494b-9fff-f6f4fb421c25`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:11:30,096::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:30,096::task::1191::Storage.TaskManager.Task::(prepare) Task=`bab30c10-9cf3-494b-9fff-f6f4fb421c25`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:30,096::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab30c10-9cf3-494b-9fff-f6f4fb421c25`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:11:30,096::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:11:30,097::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:11:30,097::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:11:30,097::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:11:30,097::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:11:30,097::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:11:30,097::task::993::Storage.TaskManager.Task::(_decref) Task=`bab30c10-9cf3-494b-9fff-f6f4fb421c25`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:30,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:30,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0112197 s, 91.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:30,938::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:30,938::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:30,939::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:30,940::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:30,940::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:30,941::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:30,941::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:30,942::__init__::225::IOProcess::(_processLogs) (2464) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:30,943::__init__::225::IOProcess::(_processLogs) (2464) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:30,945::__init__::225::IOProcess::(_processLogs) DEBUG|(2464) Building response >Thread-21::DEBUG::2014-08-22 10:11:30,946::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:30,946::__init__::225::IOProcess::(_processLogs) (2464) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:30,953::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:30,954::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:11:30,955::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000244876 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:30,957::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:30,957::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:30,958::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:30,960::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:30,961::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:30,961::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:30,962::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:30,962::__init__::225::IOProcess::(_processLogs) (2465) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:30,962::__init__::225::IOProcess::(_processLogs) (2465) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:30,963::__init__::225::IOProcess::(_processLogs) (2465) Building response >Thread-17::DEBUG::2014-08-22 10:11:30,963::__init__::225::IOProcess::(_processLogs) (2465) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:30,964::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:30,964::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:11:32,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:33,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011614 s, 88.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:33,270::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:33,270::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:33,270::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:33,270::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:33,271::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-22::DEBUG::2014-08-22 10:11:33,271::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:33,277::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:33,278::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-22::DEBUG::2014-08-22 10:11:33,279::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000239285 s, 2.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:33,279::__init__::225::IOProcess::(_processLogs) (2466) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:33,279::__init__::225::IOProcess::(_processLogs) (2466) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) (2466) Building response >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) (2466) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:33,280::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:33,281::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:33,281::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:33,281::__init__::225::IOProcess::(_processLogs) (2467) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:33,281::__init__::225::IOProcess::(_processLogs) (2467) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:33,281::__init__::225::IOProcess::(_processLogs) DEBUG|(2467) Building response >Thread-17::DEBUG::2014-08-22 10:11:33,282::__init__::225::IOProcess::(_processLogs) (2467) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:33,282::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:33,282::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:33,282::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:33,283::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:11:33,283::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:33,283::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:33,283::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:33,283::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) (2468) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) (2468) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) (2468) Building response >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) (2468) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:11:33,284::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:33,285::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:33,285::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:33,285::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:33,285::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:33,285::__init__::225::IOProcess::(_processLogs) (2469) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:33,285::__init__::225::IOProcess::(_processLogs) (2469) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) (2469) Building response >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) (2469) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:11:33,286::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) (2470) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) (2470) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) (2470) Building response >Thread-17::DEBUG::2014-08-22 10:11:33,287::__init__::225::IOProcess::(_processLogs) (2470) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:33,288::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:33,288::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:11:35,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:35,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116115 s, 88.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:37,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:37,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115704 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:39,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:39,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115642 s, 88.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:40,179::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:11:40,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`2277ed02-b0bc-4921-9246-d094a0e54b9f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:40,180::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:11:40,181::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:40,181::task::1191::Storage.TaskManager.Task::(prepare) Task=`2277ed02-b0bc-4921-9246-d094a0e54b9f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:40,181::task::595::Storage.TaskManager.Task::(_updateState) Task=`2277ed02-b0bc-4921-9246-d094a0e54b9f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:40,181::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:40,181::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:40,181::task::993::Storage.TaskManager.Task::(_decref) Task=`2277ed02-b0bc-4921-9246-d094a0e54b9f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:11:40,206::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:11:40,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`aea7d6b2-5826-4e3c-9ff8-abba0cbc4ff7`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:11:40,206::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:11:40,207::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`298f57a6-6e61-4eae-b6e7-1e617e112d9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:11:40,207::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:11:40,207::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:11:40,207::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`298f57a6-6e61-4eae-b6e7-1e617e112d9a`::Granted request >Thread-46::DEBUG::2014-08-22 10:11:40,207::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aea7d6b2-5826-4e3c-9ff8-abba0cbc4ff7`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:11:40,207::task::993::Storage.TaskManager.Task::(_decref) Task=`aea7d6b2-5826-4e3c-9ff8-abba0cbc4ff7`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:11:40,209::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:40,209::task::1191::Storage.TaskManager.Task::(prepare) Task=`aea7d6b2-5826-4e3c-9ff8-abba0cbc4ff7`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:40,210::task::595::Storage.TaskManager.Task::(_updateState) Task=`aea7d6b2-5826-4e3c-9ff8-abba0cbc4ff7`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:11:40,210::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:11:40,210::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:11:40,210::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:11:40,210::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:11:40,210::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:11:40,210::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:11:40,210::task::993::Storage.TaskManager.Task::(_decref) Task=`aea7d6b2-5826-4e3c-9ff8-abba0cbc4ff7`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:11:40,961::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:40,962::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:40,962::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-21::DEBUG::2014-08-22 10:11:40,963::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:40,963::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:40,970::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-21::DEBUG::2014-08-22 10:11:40,971::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000181416 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:40,971::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) (2471) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) (2471) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) (2471) Building response >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) (2471) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:40,972::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:40,973::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:40,973::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:40,973::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:40,973::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:40,974::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:40,974::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:40,974::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:40,974::__init__::225::IOProcess::(_processLogs) (2472) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:40,975::__init__::225::IOProcess::(_processLogs) (2472) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:40,975::__init__::225::IOProcess::(_processLogs) (2472) Building response >Thread-17::DEBUG::2014-08-22 10:11:40,975::__init__::225::IOProcess::(_processLogs) (2472) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:40,975::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:40,975::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:11:41,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:41,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140469 s, 72.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:43,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:43,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0144781 s, 70.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:43,292::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:43,292::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) (2473) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:43,293::__init__::225::IOProcess::(_processLogs) (2473) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:43,294::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2473) Building response >Thread-17::DEBUG::2014-08-22 10:11:43,294::__init__::225::IOProcess::(_processLogs) (2473) Queuing response >Thread-22::DEBUG::2014-08-22 10:11:43,294::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:43,295::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:43,301::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:11:43,302::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000196071 s, 2.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:43,303::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:43,304::__init__::225::IOProcess::(_processLogs) (2474) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:43,304::__init__::225::IOProcess::(_processLogs) (2474) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:43,304::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2474) Building response >Thread-17::DEBUG::2014-08-22 10:11:43,305::__init__::225::IOProcess::(_processLogs) (2474) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:43,305::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:43,305::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:43,305::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:43,305::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:11:43,305::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:43,306::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:43,306::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:43,306::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:43,306::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:43,306::__init__::225::IOProcess::(_processLogs) (2475) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:43,306::__init__::225::IOProcess::(_processLogs) (2475) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) (2475) Building response >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) (2475) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:11:43,307::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) (2476) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) (2476) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) (2476) Building response >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) (2476) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:43,308::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:43,309::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:43,309::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:43,309::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:11:43,309::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:43,309::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:43,309::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:43,310::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:43,310::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:43,310::__init__::225::IOProcess::(_processLogs) (2477) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:43,310::__init__::225::IOProcess::(_processLogs) (2477) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:43,310::__init__::225::IOProcess::(_processLogs) (2477) Building response >Thread-17::DEBUG::2014-08-22 10:11:43,310::__init__::225::IOProcess::(_processLogs) (2477) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:43,311::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:43,311::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::DEBUG::2014-08-22 10:11:43,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`088c8a7e-d5a1-4e53-830c-d27ba4b18575`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:43,406::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:11:43,407::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000181416', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000196071', 'lastCheck': '0.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:43,407::task::1191::Storage.TaskManager.Task::(prepare) Task=`088c8a7e-d5a1-4e53-830c-d27ba4b18575`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000181416', 'lastCheck': '2.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000196071', 'lastCheck': '0.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:43,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`088c8a7e-d5a1-4e53-830c-d27ba4b18575`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:43,407::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:43,407::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:43,407::task::993::Storage.TaskManager.Task::(_decref) Task=`088c8a7e-d5a1-4e53-830c-d27ba4b18575`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:45,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:45,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117632 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:47,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:47,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119134 s, 86.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:49,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:49,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116735 s, 87.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:50,258::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:11:50,262::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab050ee-e675-4737-8d94-bc57ca6de3d5`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:50,262::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:11:50,262::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:50,262::task::1191::Storage.TaskManager.Task::(prepare) Task=`bab050ee-e675-4737-8d94-bc57ca6de3d5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:11:50,262::task::595::Storage.TaskManager.Task::(_updateState) Task=`bab050ee-e675-4737-8d94-bc57ca6de3d5`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:50,262::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:50,262::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:50,263::task::993::Storage.TaskManager.Task::(_decref) Task=`bab050ee-e675-4737-8d94-bc57ca6de3d5`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:11:50,302::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:11:50,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf55254a-318c-46ea-bff6-0b59e75a12bd`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:11:50,302::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:11:50,302::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b75dc2d9-fb3d-4d01-9288-e93a24daf5b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:11:50,302::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:11:50,303::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:11:50,303::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b75dc2d9-fb3d-4d01-9288-e93a24daf5b6`::Granted request >Thread-46::DEBUG::2014-08-22 10:11:50,303::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cf55254a-318c-46ea-bff6-0b59e75a12bd`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:11:50,303::task::993::Storage.TaskManager.Task::(_decref) Task=`cf55254a-318c-46ea-bff6-0b59e75a12bd`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:11:50,306::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:50,306::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf55254a-318c-46ea-bff6-0b59e75a12bd`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:11:50,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf55254a-318c-46ea-bff6-0b59e75a12bd`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:11:50,307::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:11:50,307::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:11:50,307::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:11:50,307::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:11:50,307::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:11:50,307::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:11:50,307::task::993::Storage.TaskManager.Task::(_decref) Task=`cf55254a-318c-46ea-bff6-0b59e75a12bd`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:50,975::__init__::225::IOProcess::(_processLogs) (2478) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:50,976::__init__::225::IOProcess::(_processLogs) (2478) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:50,976::__init__::225::IOProcess::(_processLogs) DEBUG|(2478) Building response >Thread-17::DEBUG::2014-08-22 10:11:50,977::__init__::225::IOProcess::(_processLogs) (2478) Queuing response >Thread-21::DEBUG::2014-08-22 10:11:50,977::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:50,977::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:50,983::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:11:50,984::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000193616 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:50,984::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:50,984::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:11:50,984::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) (2479) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) (2479) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) (2479) Building response >Thread-17::DEBUG::2014-08-22 10:11:50,985::__init__::225::IOProcess::(_processLogs) (2479) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:50,986::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:11:50,987::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:11:51,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:51,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115959 s, 88.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:53,310::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) (2480) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:53,311::__init__::225::IOProcess::(_processLogs) (2480) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:53,312::__init__::225::IOProcess::(_processLogs) (2480) Queuing response >Thread-22::DEBUG::2014-08-22 10:11:53,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:11:53,313::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:53,319::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:11:53,320::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021077 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) (2481) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:11:53,321::__init__::225::IOProcess::(_processLogs) (2481) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:11:53,322::__init__::225::IOProcess::(_processLogs) DEBUG|(2481) Building response >Thread-17::DEBUG::2014-08-22 10:11:53,323::__init__::225::IOProcess::(_processLogs) (2481) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:53,323::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:53,323::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:11:53,323::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:53,323::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:11:53,323::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) (2482) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) (2482) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) (2482) Building response >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) (2482) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:53,324::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:53,325::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:53,325::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:11:53,325::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:53,325::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:53,325::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) (2483) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) (2483) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) (2483) Building response >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) (2483) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:11:53,326::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:11:53,327::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:11:53,327::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:11:53,327::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:11:53,327::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:11:53,327::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:11:53,327::__init__::225::IOProcess::(_processLogs) (2484) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:11:53,328::__init__::225::IOProcess::(_processLogs) (2484) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:11:53,328::__init__::225::IOProcess::(_processLogs) (2484) Building response >Thread-17::DEBUG::2014-08-22 10:11:53,328::__init__::225::IOProcess::(_processLogs) (2484) Queuing response >Thread-17::DEBUG::2014-08-22 10:11:53,328::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:11:53,328::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:11:53,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:53,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116456 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:55,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:55,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137584 s, 74.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:11:57,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:57,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116642 s, 87.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:11:58,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad7a3f35-bfe9-45b5-af2d-502c6e904056`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:11:58,599::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:11:58,600::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193616', 'lastCheck': '7.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021077', 'lastCheck': '5.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:58,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad7a3f35-bfe9-45b5-af2d-502c6e904056`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000193616', 'lastCheck': '7.6', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00021077', 'lastCheck': '5.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:11:58,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad7a3f35-bfe9-45b5-af2d-502c6e904056`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:11:58,600::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:11:58,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:11:58,600::task::993::Storage.TaskManager.Task::(_decref) Task=`ad7a3f35-bfe9-45b5-af2d-502c6e904056`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:11:59,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:11:59,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114825 s, 89.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:00,429::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:12:00,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9f7d18b-c5a0-45d5-8201-0f73016f420a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:00,430::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:12:00,430::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:00,430::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9f7d18b-c5a0-45d5-8201-0f73016f420a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:00,430::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9f7d18b-c5a0-45d5-8201-0f73016f420a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:00,431::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:00,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:00,431::task::993::Storage.TaskManager.Task::(_decref) Task=`b9f7d18b-c5a0-45d5-8201-0f73016f420a`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:12:00,479::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:12:00,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`c297bfa2-7366-4e81-b231-f838b76bf075`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:12:00,480::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:12:00,480::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3d9b10ae-79a8-43c3-9728-b5eea1641161`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:12:00,480::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:12:00,480::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:12:00,480::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3d9b10ae-79a8-43c3-9728-b5eea1641161`::Granted request >Thread-46::DEBUG::2014-08-22 10:12:00,481::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c297bfa2-7366-4e81-b231-f838b76bf075`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:12:00,481::task::993::Storage.TaskManager.Task::(_decref) Task=`c297bfa2-7366-4e81-b231-f838b76bf075`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:12:00,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:00,483::task::1191::Storage.TaskManager.Task::(prepare) Task=`c297bfa2-7366-4e81-b231-f838b76bf075`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:00,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`c297bfa2-7366-4e81-b231-f838b76bf075`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:12:00,483::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:12:00,483::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:12:00,483::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:12:00,484::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:12:00,484::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:12:00,484::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:12:00,484::task::993::Storage.TaskManager.Task::(_decref) Task=`c297bfa2-7366-4e81-b231-f838b76bf075`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:12:00,987::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:00,987::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:00,987::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:00,988::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:00,988::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:00,988::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:00,988::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:00,988::__init__::225::IOProcess::(_processLogs) (2485) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:00,988::__init__::225::IOProcess::(_processLogs) (2485) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:00,989::__init__::225::IOProcess::(_processLogs) DEBUG|(2485) Building response >Thread-17::DEBUG::2014-08-22 10:12:00,989::__init__::225::IOProcess::(_processLogs) (2485) Queuing response >Thread-21::DEBUG::2014-08-22 10:12:00,989::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:00,990::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:00,997::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:12:00,997::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000184402 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:00,998::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:00,998::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:00,998::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) (2486) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) (2486) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) (2486) Building response >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) (2486) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:00,999::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:01,000::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:12:01,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:01,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119697 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:03,332::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) (2487) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) (2487) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) (2487) Building response >Thread-17::DEBUG::2014-08-22 10:12:03,333::__init__::225::IOProcess::(_processLogs) (2487) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:03,334::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 10:12:03,335::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:03,335::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:12:03,341::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000230807 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:03,342::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:03,342::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:03,342::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:03,342::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:03,342::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:03,342::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:03,343::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:03,343::__init__::225::IOProcess::(_processLogs) (2488) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:03,343::__init__::225::IOProcess::(_processLogs) (2488) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:03,343::__init__::225::IOProcess::(_processLogs) (2488) Building response >Thread-17::DEBUG::2014-08-22 10:12:03,343::__init__::225::IOProcess::(_processLogs) (2488) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:03,343::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:03,344::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:03,344::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:03,345::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:12:03,345::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:03,345::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:03,345::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:03,345::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:03,345::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) (2489) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) (2489) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) (2489) Building response >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) (2489) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:12:03,346::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:03,347::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:03,347::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:03,347::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:03,347::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:03,347::__init__::225::IOProcess::(_processLogs) (2490) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:03,347::__init__::225::IOProcess::(_processLogs) (2490) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) (2490) Building response >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) (2490) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:12:03,348::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:03,349::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:03,349::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:03,349::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:03,349::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:03,349::__init__::225::IOProcess::(_processLogs) (2491) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:03,349::__init__::225::IOProcess::(_processLogs) (2491) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:03,350::__init__::225::IOProcess::(_processLogs) (2491) Building response >Thread-17::DEBUG::2014-08-22 10:12:03,350::__init__::225::IOProcess::(_processLogs) (2491) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:03,350::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:03,350::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:12:03,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:03,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011872 s, 86.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:05,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:05,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117554 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:07,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:07,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167953 s, 61.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:09,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:09,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116254 s, 88.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:10,539::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:12:10,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`667267d1-aa8d-4a90-95ee-00c2dea1a7fa`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:10,543::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:12:10,543::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:10,543::task::1191::Storage.TaskManager.Task::(prepare) Task=`667267d1-aa8d-4a90-95ee-00c2dea1a7fa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:10,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`667267d1-aa8d-4a90-95ee-00c2dea1a7fa`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:10,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:10,544::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:10,544::task::993::Storage.TaskManager.Task::(_decref) Task=`667267d1-aa8d-4a90-95ee-00c2dea1a7fa`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:12:10,580::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:12:10,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`98f36dce-bad1-4b8b-8e1e-a7daef7c3858`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:12:10,580::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:12:10,581::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3541e3db-48f7-470b-81c2-42745ca91e02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:12:10,581::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:12:10,581::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:12:10,581::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`3541e3db-48f7-470b-81c2-42745ca91e02`::Granted request >Thread-46::DEBUG::2014-08-22 10:12:10,581::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`98f36dce-bad1-4b8b-8e1e-a7daef7c3858`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:12:10,581::task::993::Storage.TaskManager.Task::(_decref) Task=`98f36dce-bad1-4b8b-8e1e-a7daef7c3858`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:12:10,584::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:10,584::task::1191::Storage.TaskManager.Task::(prepare) Task=`98f36dce-bad1-4b8b-8e1e-a7daef7c3858`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:10,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`98f36dce-bad1-4b8b-8e1e-a7daef7c3858`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:12:10,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:12:10,585::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:12:10,585::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:12:10,585::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:12:10,585::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:12:10,585::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:12:10,585::task::993::Storage.TaskManager.Task::(_decref) Task=`98f36dce-bad1-4b8b-8e1e-a7daef7c3858`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:12:11,001::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:11,001::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:11,001::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:11,001::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:11,002::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:11,002::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:11,002::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:11,002::__init__::225::IOProcess::(_processLogs) (2492) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:11,002::__init__::225::IOProcess::(_processLogs) (2492) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:11,002::__init__::225::IOProcess::(_processLogs) DEBUG|(2492) Building response >Thread-21::DEBUG::2014-08-22 10:12:11,003::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:11,003::__init__::225::IOProcess::(_processLogs) (2492) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:11,010::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:11,010::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:12:11,012::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000852798 s, 466 kB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:11,012::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:11,012::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:11,012::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:11,012::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:11,012::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:11,013::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:11,013::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:11,013::__init__::225::IOProcess::(_processLogs) (2493) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:11,013::__init__::225::IOProcess::(_processLogs) (2493) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:11,013::__init__::225::IOProcess::(_processLogs) (2493) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:11,014::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:11,014::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:12:11,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:11,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117696 s, 87.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:13,350::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:13,350::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:13,352::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:13,353::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:12:13,353::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:13,354::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:13,360::__init__::225::IOProcess::(_processLogs) (2494) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 10:12:13,361::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000178875 s, 3.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:13,361::__init__::225::IOProcess::(_processLogs) (2494) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:13,362::__init__::225::IOProcess::(_processLogs) (2494) Building response >Thread-17::DEBUG::2014-08-22 10:12:13,362::__init__::225::IOProcess::(_processLogs) (2494) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:13,362::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:13,362::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:13,362::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:13,363::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:13,363::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:13,363::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:13,363::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:13,363::__init__::225::IOProcess::(_processLogs) (2495) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:13,363::__init__::225::IOProcess::(_processLogs) (2495) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:13,364::__init__::225::IOProcess::(_processLogs) (2495) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:13,364::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:13,364::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:13,365::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:12:13,365::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:13,365::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:13,365::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:13,365::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) (2496) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) (2496) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) (2496) Building response >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) (2496) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:12:13,366::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) (2497) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) (2497) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) (2497) Building response >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) (2497) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:13,367::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:13,368::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:12:13,368::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:13,368::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:13,368::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:13,368::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:13,369::__init__::225::IOProcess::(_processLogs) (2498) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:13,369::__init__::225::IOProcess::(_processLogs) (2498) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:13,369::__init__::225::IOProcess::(_processLogs) (2498) Building response >Thread-17::DEBUG::2014-08-22 10:12:13,369::__init__::225::IOProcess::(_processLogs) (2498) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:13,369::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:13,369::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::DEBUG::2014-08-22 10:12:13,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e310c84-e1fd-472c-af83-e0636fb3fb83`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:13,816::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:12:13,816::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000852798', 'lastCheck': '2.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000178875', 'lastCheck': '0.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:13,816::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e310c84-e1fd-472c-af83-e0636fb3fb83`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000852798', 'lastCheck': '2.8', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000178875', 'lastCheck': '0.4', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:13,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e310c84-e1fd-472c-af83-e0636fb3fb83`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:13,816::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:13,816::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:13,817::task::993::Storage.TaskManager.Task::(_decref) Task=`4e310c84-e1fd-472c-af83-e0636fb3fb83`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:12:14,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:14,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118248 s, 86.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:16,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:16,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114729 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:18,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:18,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113897 s, 89.9 MB/s\n'; <rc> = 0 >Detector thread::DEBUG::2014-08-22 10:12:18,557::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44297 >Detector thread::DEBUG::2014-08-22 10:12:18,558::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44297 >Detector thread::DEBUG::2014-08-22 10:12:18,558::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44297 >Detector thread::DEBUG::2014-08-22 10:12:18,558::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44297) >Dummy-45::DEBUG::2014-08-22 10:12:20,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:20,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115282 s, 88.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:20,664::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:12:20,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`26f4240b-833f-4e14-a0bb-6516f23aaa8f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:20,664::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:12:20,664::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:20,664::task::1191::Storage.TaskManager.Task::(prepare) Task=`26f4240b-833f-4e14-a0bb-6516f23aaa8f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:20,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`26f4240b-833f-4e14-a0bb-6516f23aaa8f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:20,665::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:20,665::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:20,665::task::993::Storage.TaskManager.Task::(_decref) Task=`26f4240b-833f-4e14-a0bb-6516f23aaa8f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:12:20,690::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:12:20,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`c75b5a91-8c82-42c4-bd18-4bafccbbacca`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:12:20,690::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:12:20,691::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6f6ed366-ac74-49f5-9f4f-d4dfe91ee7b7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:12:20,691::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:12:20,691::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:12:20,691::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`6f6ed366-ac74-49f5-9f4f-d4dfe91ee7b7`::Granted request >Thread-46::DEBUG::2014-08-22 10:12:20,691::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c75b5a91-8c82-42c4-bd18-4bafccbbacca`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:12:20,691::task::993::Storage.TaskManager.Task::(_decref) Task=`c75b5a91-8c82-42c4-bd18-4bafccbbacca`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:12:20,694::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:20,694::task::1191::Storage.TaskManager.Task::(prepare) Task=`c75b5a91-8c82-42c4-bd18-4bafccbbacca`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:20,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`c75b5a91-8c82-42c4-bd18-4bafccbbacca`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:12:20,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:12:20,694::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:12:20,694::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:12:20,695::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:12:20,695::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:12:20,695::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:12:20,695::task::993::Storage.TaskManager.Task::(_decref) Task=`c75b5a91-8c82-42c4-bd18-4bafccbbacca`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) (2499) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:21,015::__init__::225::IOProcess::(_processLogs) (2499) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:21,016::__init__::225::IOProcess::(_processLogs) (2499) Queuing response >Thread-21::DEBUG::2014-08-22 10:12:21,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:21,017::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:21,023::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:12:21,024::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000286557 s, 1.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) (2500) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:21,025::__init__::225::IOProcess::(_processLogs) (2500) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:21,026::__init__::225::IOProcess::(_processLogs) (2500) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:21,027::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:21,027::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:12:22,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:22,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115506 s, 88.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) (2501) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:23,374::__init__::225::IOProcess::(_processLogs) (2501) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:23,375::__init__::225::IOProcess::(_processLogs) (2501) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:23,376::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:12:23,376::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:23,376::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:12:23,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000227956 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:23,383::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:23,383::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:23,384::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:23,384::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:23,384::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:23,384::__init__::225::IOProcess::(_processLogs) (2502) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:23,384::__init__::225::IOProcess::(_processLogs) (2502) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:23,385::__init__::225::IOProcess::(_processLogs) (2502) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:23,385::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:23,385::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:23,385::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:12:23,385::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:23,386::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:23,386::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:23,386::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:23,386::__init__::225::IOProcess::(_processLogs) (2503) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:23,386::__init__::225::IOProcess::(_processLogs) (2503) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:23,386::__init__::225::IOProcess::(_processLogs) (2503) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:23,387::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:23,387::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:23,387::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:12:23,387::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:23,387::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) (2504) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) (2504) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) (2504) Building response >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) (2504) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:12:23,388::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:23,389::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:23,389::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:23,389::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:23,389::__init__::225::IOProcess::(_processLogs) (2505) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:23,389::__init__::225::IOProcess::(_processLogs) (2505) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:23,389::__init__::225::IOProcess::(_processLogs) (2505) Building response >Thread-17::DEBUG::2014-08-22 10:12:23,390::__init__::225::IOProcess::(_processLogs) (2505) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:23,390::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:23,390::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:12:24,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:24,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119154 s, 85.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:26,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:26,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116445 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:28,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:28,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011499 s, 89.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:29,077::task::595::Storage.TaskManager.Task::(_updateState) Task=`f948285a-0a30-4a24-9f80-60bebc8b7fc3`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:29,078::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:12:29,078::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000286557', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000227956', 'lastCheck': '5.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:29,078::task::1191::Storage.TaskManager.Task::(prepare) Task=`f948285a-0a30-4a24-9f80-60bebc8b7fc3`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000286557', 'lastCheck': '8.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000227956', 'lastCheck': '5.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:29,078::task::595::Storage.TaskManager.Task::(_updateState) Task=`f948285a-0a30-4a24-9f80-60bebc8b7fc3`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:29,078::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:29,078::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:29,078::task::993::Storage.TaskManager.Task::(_decref) Task=`f948285a-0a30-4a24-9f80-60bebc8b7fc3`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:12:30,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:30,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113493 s, 90.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:30,753::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:12:30,753::task::595::Storage.TaskManager.Task::(_updateState) Task=`cda89f3e-d4e0-498b-906d-3dea53ff4488`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:30,753::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:12:30,753::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:30,754::task::1191::Storage.TaskManager.Task::(prepare) Task=`cda89f3e-d4e0-498b-906d-3dea53ff4488`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:30,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`cda89f3e-d4e0-498b-906d-3dea53ff4488`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:30,754::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:30,754::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:30,754::task::993::Storage.TaskManager.Task::(_decref) Task=`cda89f3e-d4e0-498b-906d-3dea53ff4488`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:12:30,778::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:12:30,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`880b513f-ea0b-424d-b04d-4042ff079a38`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:12:30,778::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:12:30,778::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b2b1f20e-0392-41f3-ac18-8509303923d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:12:30,778::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:12:30,778::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:12:30,778::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`b2b1f20e-0392-41f3-ac18-8509303923d9`::Granted request >Thread-46::DEBUG::2014-08-22 10:12:30,779::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`880b513f-ea0b-424d-b04d-4042ff079a38`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:12:30,779::task::993::Storage.TaskManager.Task::(_decref) Task=`880b513f-ea0b-424d-b04d-4042ff079a38`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:12:30,781::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:30,781::task::1191::Storage.TaskManager.Task::(prepare) Task=`880b513f-ea0b-424d-b04d-4042ff079a38`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:30,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`880b513f-ea0b-424d-b04d-4042ff079a38`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:12:30,781::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:12:30,781::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:12:30,781::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:12:30,781::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:12:30,781::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:12:30,781::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:12:30,782::task::993::Storage.TaskManager.Task::(_decref) Task=`880b513f-ea0b-424d-b04d-4042ff079a38`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:12:31,027::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:31,027::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:31,027::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:31,027::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:31,027::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:31,028::__init__::225::IOProcess::(_processLogs) (2506) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:31,028::__init__::225::IOProcess::(_processLogs) (2506) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:31,028::__init__::225::IOProcess::(_processLogs) (2506) Queuing response >Thread-21::DEBUG::2014-08-22 10:12:31,029::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:31,029::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:31,036::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:12:31,036::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000252 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:31,037::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:31,037::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:31,037::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:31,037::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:31,037::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:31,038::__init__::225::IOProcess::(_processLogs) (2507) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:31,038::__init__::225::IOProcess::(_processLogs) (2507) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:31,038::__init__::225::IOProcess::(_processLogs) (2507) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:31,039::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:31,039::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:12:32,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:32,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170702 s, 60.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:33,390::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:33,391::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:33,391::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:33,391::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:33,391::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:33,391::__init__::225::IOProcess::(_processLogs) (2508) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:33,391::__init__::225::IOProcess::(_processLogs) (2508) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:33,392::__init__::225::IOProcess::(_processLogs) (2508) Queuing response >Thread-22::DEBUG::2014-08-22 10:12:33,392::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:33,392::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:33,399::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:12:33,400::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000268333 s, 2.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) (2509) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:33,401::__init__::225::IOProcess::(_processLogs) (2509) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:33,402::__init__::225::IOProcess::(_processLogs) (2509) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:33,403::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:33,403::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:33,403::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:12:33,403::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:33,403::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) (2510) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) (2510) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) (2510) Building response >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) (2510) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:12:33,404::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:33,405::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:33,405::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:33,405::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:33,405::__init__::225::IOProcess::(_processLogs) (2511) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:33,405::__init__::225::IOProcess::(_processLogs) (2511) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:33,405::__init__::225::IOProcess::(_processLogs) (2511) Building response >Thread-17::DEBUG::2014-08-22 10:12:33,406::__init__::225::IOProcess::(_processLogs) (2511) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:33,406::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:33,406::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:33,406::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:12:33,406::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:33,406::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:33,407::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:33,407::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:33,407::__init__::225::IOProcess::(_processLogs) (2512) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:33,407::__init__::225::IOProcess::(_processLogs) (2512) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:33,407::__init__::225::IOProcess::(_processLogs) (2512) Building response >Thread-17::DEBUG::2014-08-22 10:12:33,407::__init__::225::IOProcess::(_processLogs) (2512) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:33,408::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:33,408::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:12:34,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:34,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114034 s, 89.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:36,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:36,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113767 s, 90.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:38,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:38,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113842 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:40,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:40,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117471 s, 87.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:40,831::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:12:40,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd0097d0-694e-40f3-b35b-4103f8d95828`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:40,832::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:12:40,832::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:40,832::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd0097d0-694e-40f3-b35b-4103f8d95828`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:40,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd0097d0-694e-40f3-b35b-4103f8d95828`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:40,832::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:40,832::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:40,832::task::993::Storage.TaskManager.Task::(_decref) Task=`dd0097d0-694e-40f3-b35b-4103f8d95828`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:12:40,872::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:12:40,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`e546c7ff-c513-433e-85af-73122ae4fa9b`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:12:40,872::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:12:40,873::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4caa5a03-aba0-4c58-93f7-f20efd3097c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:12:40,873::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:12:40,873::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:12:40,873::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`4caa5a03-aba0-4c58-93f7-f20efd3097c8`::Granted request >Thread-46::DEBUG::2014-08-22 10:12:40,873::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e546c7ff-c513-433e-85af-73122ae4fa9b`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:12:40,873::task::993::Storage.TaskManager.Task::(_decref) Task=`e546c7ff-c513-433e-85af-73122ae4fa9b`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:12:40,875::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:40,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`e546c7ff-c513-433e-85af-73122ae4fa9b`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:40,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`e546c7ff-c513-433e-85af-73122ae4fa9b`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:12:40,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:12:40,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:12:40,876::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:12:40,876::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:12:40,876::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:12:40,876::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:12:40,876::task::993::Storage.TaskManager.Task::(_decref) Task=`e546c7ff-c513-433e-85af-73122ae4fa9b`::ref 0 aborting False >Thread-21::DEBUG::2014-08-22 10:12:41,039::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) (2513) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:12:41,041::__init__::225::IOProcess::(_processLogs) (2513) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:12:41,042::__init__::225::IOProcess::(_processLogs) DEBUG|(2513) Building response >Thread-17::DEBUG::2014-08-22 10:12:41,043::__init__::225::IOProcess::(_processLogs) (2513) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,043::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:41,044::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 10:12:41,044::__init__::225::IOProcess::(_processLogs) Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,045::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 10:12:41,046::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,047::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,047::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,048::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,048::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,049::__init__::225::IOProcess::(_processLogs) (2514) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:12:41,050::__init__::225::IOProcess::(_processLogs) (2514) Got request for method 'glob' >Thread-21::DEBUG::2014-08-22 10:12:41,050::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07 >Thread-17::DEBUG::2014-08-22 10:12:41,051::__init__::225::IOProcess::(_processLogs) DEBUG|(2514) Building response >Thread-17::DEBUG::2014-08-22 10:12:41,051::__init__::225::IOProcess::(_processLogs) (2514) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,051::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:41,051::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) Message size is 170 >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) (2515) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 10:12:41,052::__init__::225::IOProcess::(_processLogs) (2515) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 10:12:41,079::__init__::225::IOProcess::(_processLogs) DEBUG|(2515) Building response >Thread-17::DEBUG::2014-08-22 10:12:41,079::__init__::225::IOProcess::(_processLogs) (2515) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,079::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 10:12:41,080::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 10:12:41,080::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:41,080::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) Message size is 194 >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) (2516) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 10:12:41,081::__init__::225::IOProcess::(_processLogs) (2516) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 10:12:41,082::__init__::225::IOProcess::(_processLogs) DEBUG|(2516) Building response >Thread-21::DEBUG::2014-08-22 10:12:41,082::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246,00000002-0002-0002-0002-00000000002a', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/iso', 'ROLE=Regular', 'SDUUID=7c4cb940-6043-48fa-88f8-ab4762019c07', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=6612de3876faf1e2a4bb3fc28ecf17781ac3bde1'] >Thread-17::DEBUG::2014-08-22 10:12:41,082::__init__::225::IOProcess::(_processLogs) (2516) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,083::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:41,083::__init__::225::IOProcess::(_processLogs) Sending response sized 604 >Thread-17::DEBUG::2014-08-22 10:12:41,083::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,083::__init__::225::IOProcess::(_processLogs) Message size is 187 >Thread-17::DEBUG::2014-08-22 10:12:41,083::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) (2517) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) (2517) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) (2517) Building response >Thread-17::DEBUG::2014-08-22 10:12:41,084::__init__::225::IOProcess::(_processLogs) (2517) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Message size is 181 >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,085::__init__::225::IOProcess::(_processLogs) (2518) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:12:41,086::__init__::225::IOProcess::(_processLogs) (2518) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:12:41,086::__init__::225::IOProcess::(_processLogs) DEBUG|(2518) Building response >Thread-21::DEBUG::2014-08-22 10:12:41,086::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 10:12:41,086::__init__::225::IOProcess::(_processLogs) (2518) Queuing response >Thread-21::INFO::2014-08-22 10:12:41,086::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_imageNS already registered >Thread-17::DEBUG::2014-08-22 10:12:41,087::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-21::INFO::2014-08-22 10:12:41,087::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7c4cb940-6043-48fa-88f8-ab4762019c07_volumeNS already registered >Thread-17::DEBUG::2014-08-22 10:12:41,087::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 10:12:41,088::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,088::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:41,088::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,088::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:12:41,089::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:41,089::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,095::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:12:41,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000236558 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:41,096::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,097::__init__::225::IOProcess::(_processLogs) (2519) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:41,097::__init__::225::IOProcess::(_processLogs) (2519) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:41,097::__init__::225::IOProcess::(_processLogs) (2519) Building response >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) (2519) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:41,098::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:41,099::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:41,099::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:41,099::__init__::225::IOProcess::(_processLogs) (2520) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:41,099::__init__::225::IOProcess::(_processLogs) (2520) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:41,099::__init__::225::IOProcess::(_processLogs) DEBUG|(2520) Building response >Thread-17::DEBUG::2014-08-22 10:12:41,100::__init__::225::IOProcess::(_processLogs) (2520) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:41,100::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:41,100::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:12:42,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:42,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115276 s, 88.8 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-08-22 10:12:43,407::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 10:12:43,410::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,410::__init__::225::IOProcess::(_processLogs) Message size is 141 >Thread-17::DEBUG::2014-08-22 10:12:43,410::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,411::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,411::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,411::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,411::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,411::__init__::225::IOProcess::(_processLogs) (2521) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) (2521) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) (2521) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) (2521) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) Sending response sized 191 >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) Message size is 143 >Thread-17::DEBUG::2014-08-22 10:12:43,412::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,413::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,414::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,415::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,416::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,417::__init__::225::IOProcess::(_processLogs) (2522) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:12:43,418::__init__::225::IOProcess::(_processLogs) (2522) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:12:43,419::__init__::225::IOProcess::(_processLogs) (2522) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,420::__init__::225::IOProcess::(_processLogs) (2522) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,421::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,422::__init__::225::IOProcess::(_processLogs) Sending response sized 193 >Thread-22::DEBUG::2014-08-22 10:12:43,423::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6 >Thread-17::DEBUG::2014-08-22 10:12:43,423::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,423::__init__::225::IOProcess::(_processLogs) Message size is 172 >Thread-17::DEBUG::2014-08-22 10:12:43,423::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,424::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,424::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,424::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,424::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,424::__init__::225::IOProcess::(_processLogs) (2523) Finding callback 'touch'... >Thread-17::DEBUG::2014-08-22 10:12:43,424::__init__::225::IOProcess::(_processLogs) (2523) Got request for method 'touch' >Thread-17::DEBUG::2014-08-22 10:12:43,453::__init__::225::IOProcess::(_processLogs) DEBUG|(2523) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,454::__init__::225::IOProcess::(_processLogs) (2523) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,454::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-22::DEBUG::2014-08-22 10:12:43,454::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-17::DEBUG::2014-08-22 10:12:43,455::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:43,455::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,455::__init__::225::IOProcess::(_processLogs) Message size is 196 >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) (2524) Finding callback 'readfile'... >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) (2524) Got request for method 'readfile' >Thread-17::DEBUG::2014-08-22 10:12:43,456::__init__::225::IOProcess::(_processLogs) DEBUG|(2524) Building response >Thread-22::DEBUG::2014-08-22 10:12:43,457::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Global_DC_1', 'POOL_DOMAINS=7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=7', 'POOL_UUID=da0e21b4-cba8-40bf-a5cd-3a50c2199246', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/nfs01', 'ROLE=Master', 'SDUUID=49f79085-53a0-4d45-8e3e-610bdf644da6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9ee161cb8cc5cec946e7b6643fb541923ea3ffcc'] >Thread-17::DEBUG::2014-08-22 10:12:43,457::__init__::225::IOProcess::(_processLogs) (2524) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,459::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,459::__init__::225::IOProcess::(_processLogs) Sending response sized 832 >Thread-17::DEBUG::2014-08-22 10:12:43,459::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,459::__init__::225::IOProcess::(_processLogs) Message size is 189 >Thread-17::DEBUG::2014-08-22 10:12:43,459::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) (2525) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) (2525) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:43,460::__init__::225::IOProcess::(_processLogs) (2525) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) (2525) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Message size is 183 >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,461::__init__::225::IOProcess::(_processLogs) (2526) Finding callback 'glob'... >Thread-17::DEBUG::2014-08-22 10:12:43,462::__init__::225::IOProcess::(_processLogs) (2526) Got request for method 'glob' >Thread-17::DEBUG::2014-08-22 10:12:43,462::__init__::225::IOProcess::(_processLogs) DEBUG|(2526) Building response >Thread-22::DEBUG::2014-08-22 10:12:43,462::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-17::DEBUG::2014-08-22 10:12:43,462::__init__::225::IOProcess::(_processLogs) (2526) Queuing response >Thread-22::INFO::2014-08-22 10:12:43,463::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_imageNS already registered >Thread-17::DEBUG::2014-08-22 10:12:43,463::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::INFO::2014-08-22 10:12:43,463::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 49f79085-53a0-4d45-8e3e-610bdf644da6_volumeNS already registered >Thread-17::DEBUG::2014-08-22 10:12:43,464::__init__::225::IOProcess::(_processLogs) Sending response sized 76 >Thread-17::DEBUG::2014-08-22 10:12:43,464::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,465::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:43,465::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,465::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-22::DEBUG::2014-08-22 10:12:43,466::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:43,466::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,473::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:12:43,474::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000258095 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:43,474::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) (2527) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) (2527) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) (2527) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) (2527) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:43,475::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) (2528) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:43,476::__init__::225::IOProcess::(_processLogs) (2528) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:43,477::__init__::225::IOProcess::(_processLogs) DEBUG|(2528) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,477::__init__::225::IOProcess::(_processLogs) (2528) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,478::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,478::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:43,478::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,478::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:12:43,478::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,478::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) (2529) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) (2529) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) (2529) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) (2529) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:43,479::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,480::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:12:43,480::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,480::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,480::__init__::225::IOProcess::(_processLogs) DEBUG|Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) (2530) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) (2530) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) (2530) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) (2530) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,481::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,482::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:43,482::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:43,482::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:12:43,482::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:43,482::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:43,482::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:43,483::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:43,483::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:43,483::__init__::225::IOProcess::(_processLogs) (2531) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:43,483::__init__::225::IOProcess::(_processLogs) (2531) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:43,483::__init__::225::IOProcess::(_processLogs) (2531) Building response >Thread-17::DEBUG::2014-08-22 10:12:43,483::__init__::225::IOProcess::(_processLogs) (2531) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:43,484::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:43,484::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-18::DEBUG::2014-08-22 10:12:44,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`a25be56f-536d-4e2b-8e07-695f0715ff2a`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:44,319::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:12:44,319::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000236558', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000258095', 'lastCheck': '0.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:44,319::task::1191::Storage.TaskManager.Task::(prepare) Task=`a25be56f-536d-4e2b-8e07-695f0715ff2a`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000236558', 'lastCheck': '3.2', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000258095', 'lastCheck': '0.8', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:44,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`a25be56f-536d-4e2b-8e07-695f0715ff2a`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:44,319::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:44,319::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:44,319::task::993::Storage.TaskManager.Task::(_decref) Task=`a25be56f-536d-4e2b-8e07-695f0715ff2a`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:12:44,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:44,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011575 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:46,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:46,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119657 s, 85.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:48,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:48,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011307 s, 90.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:50,966::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:12:50,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b87f0f8-1f9f-4fd9-a820-36110cddd5ae`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:50,967::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:12:50,967::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:50,967::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b87f0f8-1f9f-4fd9-a820-36110cddd5ae`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:12:50,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b87f0f8-1f9f-4fd9-a820-36110cddd5ae`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:50,967::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:50,968::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:50,968::task::993::Storage.TaskManager.Task::(_decref) Task=`3b87f0f8-1f9f-4fd9-a820-36110cddd5ae`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:12:51,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:51,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117843 s, 86.9 MB/s\n'; <rc> = 0 >Thread-46::DEBUG::2014-08-22 10:12:51,050::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:12:51,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8c70727-fcf6-49e0-bf6f-0b5e85e1a821`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:12:51,052::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:12:51,053::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`60b3c0e5-c637-482e-a224-eb9c7ae8b10a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:12:51,053::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:12:51,054::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:12:51,054::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`60b3c0e5-c637-482e-a224-eb9c7ae8b10a`::Granted request >Thread-46::DEBUG::2014-08-22 10:12:51,055::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8c70727-fcf6-49e0-bf6f-0b5e85e1a821`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:12:51,056::task::993::Storage.TaskManager.Task::(_decref) Task=`a8c70727-fcf6-49e0-bf6f-0b5e85e1a821`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:12:51,064::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:51,065::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8c70727-fcf6-49e0-bf6f-0b5e85e1a821`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:12:51,065::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8c70727-fcf6-49e0-bf6f-0b5e85e1a821`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:12:51,066::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:12:51,066::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:12:51,067::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:12:51,067::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:12:51,067::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:12:51,068::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:12:51,068::task::993::Storage.TaskManager.Task::(_decref) Task=`a8c70727-fcf6-49e0-bf6f-0b5e85e1a821`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) (2532) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:51,101::__init__::225::IOProcess::(_processLogs) (2532) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:51,102::__init__::225::IOProcess::(_processLogs) DEBUG|(2532) Building response >Thread-17::DEBUG::2014-08-22 10:12:51,103::__init__::225::IOProcess::(_processLogs) (2532) Queuing response >Thread-21::DEBUG::2014-08-22 10:12:51,103::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:12:51,103::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:51,110::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:12:51,110::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000231923 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) (2533) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) (2533) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:51,112::__init__::225::IOProcess::(_processLogs) (2533) Building response >Thread-17::DEBUG::2014-08-22 10:12:51,113::__init__::225::IOProcess::(_processLogs) (2533) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:51,113::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-17::DEBUG::2014-08-22 10:12:51,114::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Dummy-45::DEBUG::2014-08-22 10:12:53,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:53,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119804 s, 85.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) (2534) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) (2534) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:53,484::__init__::225::IOProcess::(_processLogs) (2534) Building response >Thread-17::DEBUG::2014-08-22 10:12:53,485::__init__::225::IOProcess::(_processLogs) (2534) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:53,485::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:53,485::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:12:53,486::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:12:53,492::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000279669 s, 2.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:12:53,493::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:53,493::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:12:53,493::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:53,494::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:53,494::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:53,494::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) (2535) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) (2535) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) (2535) Building response >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) (2535) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:12:53,495::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) (2536) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) (2536) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:53,496::__init__::225::IOProcess::(_processLogs) (2536) Building response >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) (2536) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:53,497::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) (2537) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) (2537) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) (2537) Building response >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) (2537) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:53,498::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:12:53,499::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:12:53,499::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:12:53,499::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:12:53,499::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:12:53,499::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) (2538) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) (2538) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) (2538) Building response >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) (2538) Queuing response >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:12:53,500::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:12:55,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:55,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121288 s, 84.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:57,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:57,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114648 s, 89.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:12:59,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:12:59,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113284 s, 90.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:12:59,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc76792e-0b9c-4b59-ac55-e5402ea7fde2`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:12:59,560::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:12:59,560::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000231923', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000279669', 'lastCheck': '6.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:59,560::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc76792e-0b9c-4b59-ac55-e5402ea7fde2`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000231923', 'lastCheck': '8.4', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000279669', 'lastCheck': '6.1', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:12:59,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc76792e-0b9c-4b59-ac55-e5402ea7fde2`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:12:59,560::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:12:59,560::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:12:59,560::task::993::Storage.TaskManager.Task::(_decref) Task=`dc76792e-0b9c-4b59-ac55-e5402ea7fde2`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:13:01,114::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) (2539) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:01,115::__init__::225::IOProcess::(_processLogs) (2539) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:01,116::__init__::225::IOProcess::(_processLogs) DEBUG|(2539) Building response >Thread-17::DEBUG::2014-08-22 10:13:01,116::__init__::225::IOProcess::(_processLogs) (2539) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:01,116::__init__::225::IOProcess::(_processLogs) DEBUG|Generating json... >Thread-21::DEBUG::2014-08-22 10:13:01,117::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:01,117::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:13:01,123::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.00021154 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:01,124::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:01,124::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:01,124::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:01,124::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:01,124::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:01,125::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:01,125::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:01,125::__init__::225::IOProcess::(_processLogs) (2540) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:01,125::__init__::225::IOProcess::(_processLogs) (2540) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:01,125::__init__::225::IOProcess::(_processLogs) (2540) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:01,126::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:01,126::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:13:01,128::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:13:01,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`3a2d7e9f-7393-46bb-8207-ceb1ff922db6`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:01,128::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:13:01,129::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:01,129::task::1191::Storage.TaskManager.Task::(prepare) Task=`3a2d7e9f-7393-46bb-8207-ceb1ff922db6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:01,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`3a2d7e9f-7393-46bb-8207-ceb1ff922db6`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:01,129::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:01,129::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:01,129::task::993::Storage.TaskManager.Task::(_decref) Task=`3a2d7e9f-7393-46bb-8207-ceb1ff922db6`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:13:01,164::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:13:01,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4e783bb-8c0f-40fc-bf78-8223b8902130`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:13:01,164::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:13:01,164::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2f2daed9-89fd-4eb4-8378-3dca18e782ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:13:01,164::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:13:01,165::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:13:01,165::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2f2daed9-89fd-4eb4-8378-3dca18e782ce`::Granted request >Thread-46::DEBUG::2014-08-22 10:13:01,165::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a4e783bb-8c0f-40fc-bf78-8223b8902130`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:13:01,165::task::993::Storage.TaskManager.Task::(_decref) Task=`a4e783bb-8c0f-40fc-bf78-8223b8902130`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:13:01,167::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:01,167::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4e783bb-8c0f-40fc-bf78-8223b8902130`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:01,167::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4e783bb-8c0f-40fc-bf78-8223b8902130`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:13:01,167::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:13:01,167::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:13:01,168::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:13:01,168::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:13:01,168::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:13:01,168::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:13:01,168::task::993::Storage.TaskManager.Task::(_decref) Task=`a4e783bb-8c0f-40fc-bf78-8223b8902130`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:01,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:01,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116986 s, 87.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:03,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:03,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114435 s, 89.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:03,500::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:03,500::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:03,500::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:03,501::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:03,501::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:03,501::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:03,501::__init__::225::IOProcess::(_processLogs) (2541) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:03,501::__init__::225::IOProcess::(_processLogs) (2541) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:03,501::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2541) Building response >Thread-17::DEBUG::2014-08-22 10:13:03,502::__init__::225::IOProcess::(_processLogs) (2541) Queuing response >Thread-22::DEBUG::2014-08-22 10:13:03,502::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:03,502::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:03,509::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:13:03,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00021664 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:03,510::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:03,510::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:03,511::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) (2542) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) (2542) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) (2542) Building response >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) (2542) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:03,512::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:03,513::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:03,513::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:13:03,513::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:03,513::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) (2543) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) (2543) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) (2543) Building response >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) (2543) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:13:03,514::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:03,515::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:03,515::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:03,515::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:03,515::__init__::225::IOProcess::(_processLogs) (2544) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:03,515::__init__::225::IOProcess::(_processLogs) (2544) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:03,515::__init__::225::IOProcess::(_processLogs) (2544) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) (2545) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:03,516::__init__::225::IOProcess::(_processLogs) (2545) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:03,517::__init__::225::IOProcess::(_processLogs) (2545) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:03,517::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:03,517::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:13:05,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:05,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138273 s, 74.1 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:07,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:07,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113494 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:09,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:09,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140152 s, 73.1 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:11,127::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:11,131::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-21::DEBUG::2014-08-22 10:13:11,132::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:11,132::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:11,138::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:11,139::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:11,139::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:11,139::__init__::225::IOProcess::(_processLogs) (2546) Finding callback 'statvfs'... >Thread-21::DEBUG::2014-08-22 10:13:11,140::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000252504 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:11,140::__init__::225::IOProcess::(_processLogs) (2546) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) (2546) Building response >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) (2546) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:11,141::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:11,142::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:11,142::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:11,142::__init__::225::IOProcess::(_processLogs) (2547) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:11,142::__init__::225::IOProcess::(_processLogs) (2547) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:11,143::__init__::225::IOProcess::(_processLogs) (2547) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:11,143::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:11,143::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:13:11,216::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:13:11,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`5edc1eb4-89ce-4031-98ff-c761bd37f461`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:11,216::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:13:11,216::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:11,216::task::1191::Storage.TaskManager.Task::(prepare) Task=`5edc1eb4-89ce-4031-98ff-c761bd37f461`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:11,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`5edc1eb4-89ce-4031-98ff-c761bd37f461`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:11,217::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:11,217::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:11,217::task::993::Storage.TaskManager.Task::(_decref) Task=`5edc1eb4-89ce-4031-98ff-c761bd37f461`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:13:11,251::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:13:11,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a3f169e-af1c-4478-b6dc-dc401c5ee57f`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:13:11,251::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:13:11,251::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a2ac00b0-3338-45a9-a119-f484e25c5812`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:13:11,252::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:13:11,252::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:13:11,252::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`a2ac00b0-3338-45a9-a119-f484e25c5812`::Granted request >Thread-46::DEBUG::2014-08-22 10:13:11,252::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1a3f169e-af1c-4478-b6dc-dc401c5ee57f`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:13:11,252::task::993::Storage.TaskManager.Task::(_decref) Task=`1a3f169e-af1c-4478-b6dc-dc401c5ee57f`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:13:11,255::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:11,255::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a3f169e-af1c-4478-b6dc-dc401c5ee57f`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:11,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a3f169e-af1c-4478-b6dc-dc401c5ee57f`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:13:11,255::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:13:11,255::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:13:11,255::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:13:11,255::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:13:11,256::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:13:11,256::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:13:11,256::task::993::Storage.TaskManager.Task::(_decref) Task=`1a3f169e-af1c-4478-b6dc-dc401c5ee57f`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:11,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:11,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118833 s, 86.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:13,518::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:13,518::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:13,520::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:13,521::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:13:13,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:13,521::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:13,528::__init__::225::IOProcess::(_processLogs) (2548) Finding callback 'statvfs'... >Thread-22::DEBUG::2014-08-22 10:13:13,529::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000214638 s, 2.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:13,529::__init__::225::IOProcess::(_processLogs) (2548) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:13,530::__init__::225::IOProcess::(_processLogs) (2548) Building response >Thread-17::DEBUG::2014-08-22 10:13:13,530::__init__::225::IOProcess::(_processLogs) (2548) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:13,530::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:13,530::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:13,530::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:13,530::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:13,531::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:13,531::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:13,531::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:13,531::__init__::225::IOProcess::(_processLogs) (2549) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:13,531::__init__::225::IOProcess::(_processLogs) (2549) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:13,532::__init__::225::IOProcess::(_processLogs) (2549) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:13,532::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:13,532::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:13,532::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:13:13,533::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:13,533::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:13,533::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:13,533::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) (2550) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) (2550) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) (2550) Building response >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) (2550) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:13:13,534::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) (2551) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) (2551) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) (2551) Building response >Thread-17::DEBUG::2014-08-22 10:13:13,535::__init__::225::IOProcess::(_processLogs) (2551) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:13,536::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:13,536::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:13,536::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:13:13,536::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:13,536::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:13,536::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) (2552) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) (2552) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) (2552) Building response >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) (2552) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:13,537::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:13:13,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:13,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115686 s, 88.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:13:14,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`58b78ca0-b3f9-400e-87ce-ecd77faa7758`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:14,834::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:13:14,834::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252504', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214638', 'lastCheck': '1.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:13:14,834::task::1191::Storage.TaskManager.Task::(prepare) Task=`58b78ca0-b3f9-400e-87ce-ecd77faa7758`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000252504', 'lastCheck': '3.7', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000214638', 'lastCheck': '1.3', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:13:14,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`58b78ca0-b3f9-400e-87ce-ecd77faa7758`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:14,835::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:14,835::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:14,835::task::993::Storage.TaskManager.Task::(_decref) Task=`58b78ca0-b3f9-400e-87ce-ecd77faa7758`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:15,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:15,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113369 s, 90.3 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:17,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:17,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113495 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:19,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:19,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114617 s, 89.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) (2553) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:21,144::__init__::225::IOProcess::(_processLogs) (2553) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:21,145::__init__::225::IOProcess::(_processLogs) (2553) Queuing response >Thread-21::DEBUG::2014-08-22 10:13:21,146::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:21,146::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:21,152::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:13:21,153::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000223576 s, 1.8 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) (2554) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:21,154::__init__::225::IOProcess::(_processLogs) (2554) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:21,155::__init__::225::IOProcess::(_processLogs) DEBUG|(2554) Building response >Thread-17::DEBUG::2014-08-22 10:13:21,156::__init__::225::IOProcess::(_processLogs) (2554) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:21,156::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:21,156::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:13:21,304::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:13:21,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`a450002d-132e-429c-a62a-feccbb043910`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:21,304::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:13:21,304::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:21,304::task::1191::Storage.TaskManager.Task::(prepare) Task=`a450002d-132e-429c-a62a-feccbb043910`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:21,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`a450002d-132e-429c-a62a-feccbb043910`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:21,305::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:21,305::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:21,305::task::993::Storage.TaskManager.Task::(_decref) Task=`a450002d-132e-429c-a62a-feccbb043910`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:13:21,338::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:13:21,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c5f5714-fcd1-4acd-a20a-e7c02c5989c6`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:13:21,338::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:13:21,338::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0764482c-ef82-49cd-8a16-4f4576e31a1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:13:21,338::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:13:21,339::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:13:21,339::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0764482c-ef82-49cd-8a16-4f4576e31a1d`::Granted request >Thread-46::DEBUG::2014-08-22 10:13:21,339::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8c5f5714-fcd1-4acd-a20a-e7c02c5989c6`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:13:21,339::task::993::Storage.TaskManager.Task::(_decref) Task=`8c5f5714-fcd1-4acd-a20a-e7c02c5989c6`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:13:21,341::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:21,341::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c5f5714-fcd1-4acd-a20a-e7c02c5989c6`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:21,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c5f5714-fcd1-4acd-a20a-e7c02c5989c6`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:13:21,341::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:13:21,341::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:13:21,341::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:13:21,341::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:13:21,341::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:13:21,342::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:13:21,342::task::993::Storage.TaskManager.Task::(_decref) Task=`8c5f5714-fcd1-4acd-a20a-e7c02c5989c6`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:21,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:21,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117007 s, 87.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:23,542::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:23,546::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:23,546::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-22::DEBUG::2014-08-22 10:13:23,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:23,547::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:23,553::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:23,554::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-22::DEBUG::2014-08-22 10:13:23,555::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000248943 s, 2.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:23,554::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) (2555) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) (2555) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) (2555) Building response >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) (2555) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:23,556::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:23,557::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:23,557::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:23,557::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:23,557::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:23,557::__init__::225::IOProcess::(_processLogs) (2556) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:23,557::__init__::225::IOProcess::(_processLogs) (2556) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:23,558::__init__::225::IOProcess::(_processLogs) DEBUG|(2556) Building response >Thread-17::DEBUG::2014-08-22 10:13:23,558::__init__::225::IOProcess::(_processLogs) (2556) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:23,558::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:23,558::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:23,558::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:23,559::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:13:23,559::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:23,559::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:23,559::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:23,559::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) (2557) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) (2557) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) (2557) Building response >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) (2557) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:13:23,560::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) (2558) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) (2558) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) (2558) Building response >Thread-17::DEBUG::2014-08-22 10:13:23,561::__init__::225::IOProcess::(_processLogs) (2558) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:23,562::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:23,562::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:23,562::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:23,562::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:13:23,562::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:23,562::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:23,563::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:23,563::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:23,563::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:23,563::__init__::225::IOProcess::(_processLogs) (2559) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:23,563::__init__::225::IOProcess::(_processLogs) (2559) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:23,563::__init__::225::IOProcess::(_processLogs) (2559) Building response >Thread-17::DEBUG::2014-08-22 10:13:23,564::__init__::225::IOProcess::(_processLogs) (2559) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:23,564::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:23,564::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:13:23,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:23,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113586 s, 90.2 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:25,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:25,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0128662 s, 79.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:27,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:27,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115039 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:30,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:30,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116842 s, 87.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:13:30,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`6505bf99-010e-4903-b876-f7664bacfede`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:30,047::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:13:30,048::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223576', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000248943', 'lastCheck': '6.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:13:30,048::task::1191::Storage.TaskManager.Task::(prepare) Task=`6505bf99-010e-4903-b876-f7664bacfede`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000223576', 'lastCheck': '8.9', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000248943', 'lastCheck': '6.5', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:13:30,049::task::595::Storage.TaskManager.Task::(_updateState) Task=`6505bf99-010e-4903-b876-f7664bacfede`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:30,049::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:30,049::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:30,050::task::993::Storage.TaskManager.Task::(_decref) Task=`6505bf99-010e-4903-b876-f7664bacfede`::ref 0 aborting False >Thread-17::DEBUG::2014-08-22 10:13:31,156::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:31,156::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:31,157::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-21::DEBUG::2014-08-22 10:13:31,158::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:31,158::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:31,165::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:31,166::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:13:31,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000258428 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:31,167::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:31,167::__init__::225::IOProcess::(_processLogs) (2560) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:31,167::__init__::225::IOProcess::(_processLogs) (2560) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:31,167::__init__::225::IOProcess::(_processLogs) (2560) Building response >Thread-17::DEBUG::2014-08-22 10:13:31,167::__init__::225::IOProcess::(_processLogs) (2560) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:31,168::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:31,169::__init__::225::IOProcess::(_processLogs) (2561) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:31,169::__init__::225::IOProcess::(_processLogs) (2561) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:31,169::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|(2561) Building response >Thread-17::DEBUG::2014-08-22 10:13:31,170::__init__::225::IOProcess::(_processLogs) (2561) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:31,170::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:31,170::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:13:31,417::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:13:31,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1cf3e49-56a5-43f9-9b84-5b669683a94e`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:31,417::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:13:31,417::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:31,418::task::1191::Storage.TaskManager.Task::(prepare) Task=`f1cf3e49-56a5-43f9-9b84-5b669683a94e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:31,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1cf3e49-56a5-43f9-9b84-5b669683a94e`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:31,418::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:31,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:31,418::task::993::Storage.TaskManager.Task::(_decref) Task=`f1cf3e49-56a5-43f9-9b84-5b669683a94e`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:13:31,442::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:13:31,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0c47945-0e7b-4331-ae49-c4ff70fbcc2e`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:13:31,442::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:13:31,442::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`77c1bb3a-78c4-4ef3-8b12-db49de3dad3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:13:31,442::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:13:31,443::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:13:31,443::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`77c1bb3a-78c4-4ef3-8b12-db49de3dad3a`::Granted request >Thread-46::DEBUG::2014-08-22 10:13:31,443::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d0c47945-0e7b-4331-ae49-c4ff70fbcc2e`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:13:31,443::task::993::Storage.TaskManager.Task::(_decref) Task=`d0c47945-0e7b-4331-ae49-c4ff70fbcc2e`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:13:31,445::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:31,445::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0c47945-0e7b-4331-ae49-c4ff70fbcc2e`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:31,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0c47945-0e7b-4331-ae49-c4ff70fbcc2e`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:13:31,445::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:13:31,445::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:13:31,446::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:13:31,446::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:13:31,446::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:13:31,446::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:13:31,446::task::993::Storage.TaskManager.Task::(_decref) Task=`d0c47945-0e7b-4331-ae49-c4ff70fbcc2e`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:32,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:32,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213292 s, 48.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Marshaling message...DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) (2562) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:33,564::__init__::225::IOProcess::(_processLogs) (2562) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:33,565::__init__::225::IOProcess::(_processLogs) DEBUG|(2562) Building response >Thread-22::DEBUG::2014-08-22 10:13:33,566::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:33,566::__init__::225::IOProcess::(_processLogs) (2562) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:33,572::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:33,573::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:13:33,573::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000256791 s, 2.2 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:33,574::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:33,574::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:33,574::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:33,574::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:33,575::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:33,575::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:33,575::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:33,575::__init__::225::IOProcess::(_processLogs) (2563) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:33,575::__init__::225::IOProcess::(_processLogs) (2563) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:33,575::__init__::225::IOProcess::(_processLogs) DEBUG|(2563) Building response >Thread-17::DEBUG::2014-08-22 10:13:33,576::__init__::225::IOProcess::(_processLogs) (2563) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:33,576::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:33,576::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:33,577::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:33,577::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:13:33,577::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:33,577::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:33,577::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:33,577::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) (2564) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) (2564) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) (2564) Building response >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) (2564) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:13:33,578::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) (2565) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) (2565) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:33,579::__init__::225::IOProcess::(_processLogs) (2565) Building response >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) (2565) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:33,580::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) (2566) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) (2566) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) (2566) Building response >Thread-17::DEBUG::2014-08-22 10:13:33,581::__init__::225::IOProcess::(_processLogs) (2566) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:33,582::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:33,582::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:13:34,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:34,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116372 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:36,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:36,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113915 s, 89.9 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:38,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:38,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115656 s, 88.5 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:40,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:40,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115048 s, 89.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:41,171::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:41,171::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:41,171::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:41,171::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-21::DEBUG::2014-08-22 10:13:41,172::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:41,172::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:41,179::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-08-22 10:13:41,180::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000236517 s, 1.7 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:41,180::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:41,181::__init__::225::IOProcess::(_processLogs) (2567) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:41,181::__init__::225::IOProcess::(_processLogs) (2567) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:41,181::__init__::225::IOProcess::(_processLogs) (2567) Building response >Thread-17::DEBUG::2014-08-22 10:13:41,181::__init__::225::IOProcess::(_processLogs) (2567) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:41,181::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:41,181::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) DEBUG|Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) (2568) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:41,182::__init__::225::IOProcess::(_processLogs) (2568) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:41,183::__init__::225::IOProcess::(_processLogs) DEBUG|(2568) Building response >Thread-17::DEBUG::2014-08-22 10:13:41,184::__init__::225::IOProcess::(_processLogs) (2568) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:41,184::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:41,184::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:13:41,495::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:13:41,495::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb17dda7-6134-400f-898e-83936c974744`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:41,496::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:13:41,496::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:41,496::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb17dda7-6134-400f-898e-83936c974744`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:41,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb17dda7-6134-400f-898e-83936c974744`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:41,497::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:41,497::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:41,497::task::993::Storage.TaskManager.Task::(_decref) Task=`bb17dda7-6134-400f-898e-83936c974744`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:13:41,520::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:13:41,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`339b77e2-4f76-4c06-9840-062b8a583b26`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:13:41,520::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:13:41,520::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2ac2cbbe-0a88-4236-a2a0-9f4a42e02ae0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:13:41,520::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:13:41,521::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:13:41,521::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`2ac2cbbe-0a88-4236-a2a0-9f4a42e02ae0`::Granted request >Thread-46::DEBUG::2014-08-22 10:13:41,521::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`339b77e2-4f76-4c06-9840-062b8a583b26`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:13:41,521::task::993::Storage.TaskManager.Task::(_decref) Task=`339b77e2-4f76-4c06-9840-062b8a583b26`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:13:41,523::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:41,523::task::1191::Storage.TaskManager.Task::(prepare) Task=`339b77e2-4f76-4c06-9840-062b8a583b26`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:41,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`339b77e2-4f76-4c06-9840-062b8a583b26`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:13:41,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:13:41,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:13:41,523::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:13:41,523::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:13:41,524::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:13:41,524::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:13:41,524::task::993::Storage.TaskManager.Task::(_decref) Task=`339b77e2-4f76-4c06-9840-062b8a583b26`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:42,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:42,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118366 s, 86.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:43,581::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) (2569) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) (2569) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) (2569) Building response >Thread-17::DEBUG::2014-08-22 10:13:43,582::__init__::225::IOProcess::(_processLogs) (2569) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:43,583::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:43,583::__init__::225::IOProcess::(_processLogs) DEBUG|Sending response sized 329 >Thread-22::DEBUG::2014-08-22 10:13:43,584::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-08-22 10:13:43,590::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.000221746 s, 2.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:43,590::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:43,591::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:43,591::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:43,591::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) (2570) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) (2570) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) (2570) Building response >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) (2570) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:43,592::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:43,593::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:43,593::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:43,593::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:13:43,593::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:43,593::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) (2571) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) (2571) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) (2571) Building response >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) (2571) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:43,594::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:43,595::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:13:43,595::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:43,595::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:43,595::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:43,595::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) (2572) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) (2572) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) (2572) Building response >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) (2572) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:13:43,596::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:43,597::__init__::225::IOProcess::(_processLogs) DEBUG|Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:43,597::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) (2573) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) (2573) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) (2573) Building response >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) (2573) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:43,598::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:13:44,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:44,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114677 s, 89.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:13:45,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d796b2a-7dd8-4b42-ae6a-ea5b8d480ef8`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:45,320::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:13:45,320::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000236517', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221746', 'lastCheck': '1.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:13:45,320::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d796b2a-7dd8-4b42-ae6a-ea5b8d480ef8`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000236517', 'lastCheck': '4.1', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000221746', 'lastCheck': '1.7', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:13:45,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d796b2a-7dd8-4b42-ae6a-ea5b8d480ef8`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:45,320::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:45,320::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:45,320::task::993::Storage.TaskManager.Task::(_decref) Task=`2d796b2a-7dd8-4b42-ae6a-ea5b8d480ef8`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:46,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:46,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116583 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:48,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:48,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115291 s, 88.8 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:50,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:50,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113837 s, 90.0 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:51,184::__init__::225::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) (2574) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:51,185::__init__::225::IOProcess::(_processLogs) (2574) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:51,186::__init__::225::IOProcess::(_processLogs) (2574) Queuing response >Thread-21::DEBUG::2014-08-22 10:13:51,186::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:51,187::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:51,193::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:13:51,194::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000204272 s, 1.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) (2575) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:51,195::__init__::225::IOProcess::(_processLogs) (2575) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:51,196::__init__::225::IOProcess::(_processLogs) (2575) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:51,197::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:51,197::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:13:51,582::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:13:51,582::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b768e83-83cf-46c8-8407-203fe45f1bcb`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:13:51,582::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:13:51,583::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:51,583::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b768e83-83cf-46c8-8407-203fe45f1bcb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:13:51,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b768e83-83cf-46c8-8407-203fe45f1bcb`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:13:51,583::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:13:51,583::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:13:51,583::task::993::Storage.TaskManager.Task::(_decref) Task=`4b768e83-83cf-46c8-8407-203fe45f1bcb`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:13:51,619::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:13:51,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`a92de2a9-31ba-4cdc-8690-6759c1610177`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:13:51,620::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:13:51,620::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f3cd64be-745f-4704-a9e7-7dd455042a0d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:13:51,620::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:13:51,620::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:13:51,620::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`f3cd64be-745f-4704-a9e7-7dd455042a0d`::Granted request >Thread-46::DEBUG::2014-08-22 10:13:51,620::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a92de2a9-31ba-4cdc-8690-6759c1610177`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:13:51,621::task::993::Storage.TaskManager.Task::(_decref) Task=`a92de2a9-31ba-4cdc-8690-6759c1610177`::ref 1 aborting False >Thread-46::INFO::2014-08-22 10:13:51,627::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:51,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`a92de2a9-31ba-4cdc-8690-6759c1610177`::finished: {'info': {'name': 'Global_DC_1', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 7, 'domains': '7c4cb940-6043-48fa-88f8-ab4762019c07:Active,49f79085-53a0-4d45-8e3e-610bdf644da6:Active,76b88f40-ff55-49c3-921c-64e0a4c14418:Attached', 'master_uuid': u'49f79085-53a0-4d45-8e3e-610bdf644da6', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': u'/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '422758580224', 'version': 0}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'status': 'Active', 'diskfree': '107358453760', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}, '76b88f40-ff55-49c3-921c-64e0a4c14418': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-46::DEBUG::2014-08-22 10:13:51,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`a92de2a9-31ba-4cdc-8690-6759c1610177`::moving from state preparing -> state finished >Thread-46::DEBUG::2014-08-22 10:13:51,629::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246': < ResourceRef 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', isValid: 'True' obj: 'None'>} >Thread-46::DEBUG::2014-08-22 10:13:51,629::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2014-08-22 10:13:51,629::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' >Thread-46::DEBUG::2014-08-22 10:13:51,630::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' (0 active users) >Thread-46::DEBUG::2014-08-22 10:13:51,630::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free, finding out if anyone is waiting for it. >Thread-46::DEBUG::2014-08-22 10:13:51,630::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246', Clearing records. >Thread-46::DEBUG::2014-08-22 10:13:51,630::task::993::Storage.TaskManager.Task::(_decref) Task=`a92de2a9-31ba-4cdc-8690-6759c1610177`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:13:52,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:52,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117168 s, 87.4 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:53,598::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:53,598::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:53,598::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:53,598::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:53,599::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:53,599::__init__::225::IOProcess::(_processLogs) (2576) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:53,599::__init__::225::IOProcess::(_processLogs) (2576) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:53,599::__init__::225::IOProcess::(_processLogs) (2576) Building response >Thread-22::DEBUG::2014-08-22 10:13:53,600::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01/49f79085-53a0-4d45-8e3e-610bdf644da6/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:13:53,600::__init__::225::IOProcess::(_processLogs) (2576) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:53,607::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-22::DEBUG::2014-08-22 10:13:53,608::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n570 bytes (570 B) copied, 0.00022556 s, 2.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:13:53,608::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:53,609::__init__::225::IOProcess::(_processLogs) Message size is 163 >Thread-17::DEBUG::2014-08-22 10:13:53,609::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:13:53,609::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:53,609::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:53,609::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:53,609::__init__::225::IOProcess::(_processLogs) (2577) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:13:53,610::__init__::225::IOProcess::(_processLogs) (2577) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:13:53,610::__init__::225::IOProcess::(_processLogs) (2577) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:53,611::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:53,611::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-17::DEBUG::2014-08-22 10:13:53,611::__init__::225::IOProcess::(_processLogs) Message size is 180 >Thread-17::DEBUG::2014-08-22 10:13:53,611::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) (2578) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) (2578) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) (2578) Building response >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) (2578) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:53,612::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:53,613::__init__::225::IOProcess::(_processLogs) Message size is 184 >Thread-17::DEBUG::2014-08-22 10:13:53,613::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:53,613::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:53,613::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:53,613::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) (2579) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) (2579) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) (2579) Building response >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) (2579) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) Message size is 186 >Thread-17::DEBUG::2014-08-22 10:13:53,614::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:13:53,615::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:13:53,615::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:13:53,615::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:13:53,615::__init__::225::IOProcess::(_processLogs) (2580) Finding callback 'access'... >Thread-17::DEBUG::2014-08-22 10:13:53,615::__init__::225::IOProcess::(_processLogs) (2580) Got request for method 'access' >Thread-17::DEBUG::2014-08-22 10:13:53,616::__init__::225::IOProcess::(_processLogs) (2580) Building response >Thread-17::DEBUG::2014-08-22 10:13:53,616::__init__::225::IOProcess::(_processLogs) (2580) Queuing response >Thread-17::DEBUG::2014-08-22 10:13:53,616::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:13:53,616::__init__::225::IOProcess::(_processLogs) Sending response sized 74 >Dummy-45::DEBUG::2014-08-22 10:13:54,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:54,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134109 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:56,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:56,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116873 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-45::DEBUG::2014-08-22 10:13:58,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:13:59,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115797 s, 88.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-08-22 10:14:00,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`507a2d26-faed-4fd2-86e4-f3e1db455f66`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:14:00,529::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-08-22 10:14:00,529::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000204272', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00022556', 'lastCheck': '6.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:14:00,529::task::1191::Storage.TaskManager.Task::(prepare) Task=`507a2d26-faed-4fd2-86e4-f3e1db455f66`::finished: {'7c4cb940-6043-48fa-88f8-ab4762019c07': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000204272', 'lastCheck': '9.3', 'valid': True}, '49f79085-53a0-4d45-8e3e-610bdf644da6': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00022556', 'lastCheck': '6.9', 'valid': True}} >Thread-18::DEBUG::2014-08-22 10:14:00,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`507a2d26-faed-4fd2-86e4-f3e1db455f66`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:14:00,529::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:14:00,529::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:14:00,529::task::993::Storage.TaskManager.Task::(_decref) Task=`507a2d26-faed-4fd2-86e4-f3e1db455f66`::ref 0 aborting False >Dummy-45::DEBUG::2014-08-22 10:14:01,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/da0e21b4-cba8-40bf-a5cd-3a50c2199246/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-45::DEBUG::2014-08-22 10:14:01,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115224 s, 88.9 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:14:01,197::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:14:01,197::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:14:01,197::__init__::225::IOProcess::(_processLogs) Queuing request... >Thread-17::DEBUG::2014-08-22 10:14:01,198::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:14:01,198::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:14:01,198::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:14:01,198::__init__::225::IOProcess::(_processLogs) (2581) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:14:01,198::__init__::225::IOProcess::(_processLogs) (2581) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:14:01,199::__init__::225::IOProcess::(_processLogs) (2581) Queuing response >Thread-21::DEBUG::2014-08-22 10:14:01,199::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_iso/7c4cb940-6043-48fa-88f8-ab4762019c07/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-17::DEBUG::2014-08-22 10:14:01,199::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:14:01,206::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-21::DEBUG::2014-08-22 10:14:01,207::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n397 bytes (397 B) copied, 0.000253818 s, 1.6 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-08-22 10:14:01,207::__init__::225::IOProcess::(_processLogs) Message size is 161 >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) Marshaling message... >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) Waiting for next request... >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) (2582) Finding callback 'statvfs'... >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) (2582) Got request for method 'statvfs' >Thread-17::DEBUG::2014-08-22 10:14:01,208::__init__::225::IOProcess::(_processLogs) (2582) Building response >Thread-17::DEBUG::2014-08-22 10:14:01,209::__init__::225::IOProcess::(_processLogs) Generating json... >Thread-17::DEBUG::2014-08-22 10:14:01,210::__init__::225::IOProcess::(_processLogs) Sending response sized 329 >Thread-18::DEBUG::2014-08-22 10:14:01,698::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-18::DEBUG::2014-08-22 10:14:01,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d2bf749-ce94-402a-8515-88d0ff4e191f`::moving from state init -> state preparing >Thread-18::INFO::2014-08-22 10:14:01,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-18::INFO::2014-08-22 10:14:01,698::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:14:01,698::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d2bf749-ce94-402a-8515-88d0ff4e191f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 7}} >Thread-18::DEBUG::2014-08-22 10:14:01,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d2bf749-ce94-402a-8515-88d0ff4e191f`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-08-22 10:14:01,699::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-08-22 10:14:01,699::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-08-22 10:14:01,699::task::993::Storage.TaskManager.Task::(_decref) Task=`6d2bf749-ce94-402a-8515-88d0ff4e191f`::ref 0 aborting False >Thread-46::DEBUG::2014-08-22 10:14:01,749::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.60] >Thread-46::DEBUG::2014-08-22 10:14:01,749::task::595::Storage.TaskManager.Task::(_updateState) Task=`56b2b859-6ae7-4e34-b56d-a2e4cfd142e8`::moving from state init -> state preparing >Thread-46::INFO::2014-08-22 10:14:01,749::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='da0e21b4-cba8-40bf-a5cd-3a50c2199246', options=None) >Thread-46::DEBUG::2014-08-22 10:14:01,750::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0ce973cf-e6c8-4c6e-bd1e-0199471be61b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-46::DEBUG::2014-08-22 10:14:01,750::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' for lock type 'shared' >Thread-46::DEBUG::2014-08-22 10:14:01,750::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246' is free. Now locking as 'shared' (1 active user) >Thread-46::DEBUG::2014-08-22 10:14:01,750::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246`ReqID=`0ce973cf-e6c8-4c6e-bd1e-0199471be61b`::Granted request >Thread-46::DEBUG::2014-08-22 10:14:01,750::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`56b2b859-6ae7-4e34-b56d-a2e4cfd142e8`::_resourcesAcquired: Storage.da0e21b4-cba8-40bf-a5cd-3a50c2199246 (shared) >Thread-46::DEBUG::2014-08-22 10:14:01,750::task::993::Storage.TaskManager.Task::(_decref) Task=`56b2b859-6ae7-4e34-b56d-a2e4cfd142e8`::ref 1 aborting False >